kobotoolbox / kpi / 22689698406

81%

master: 76%

push

github

fix(reports): sanitize non-finite floats to prevent invalid JSON responses DEV-1797 (#6779)

### 📣 Summary

Fixed an issue where the reports page would fail to load when a decimal
field contained a non-finite value (e.g. `Infinity` or `NaN`) in a
submission.

Also fixes the statistics computation so that submissions containing
non-finite decimal values are treated as **not provided**, rather than
corrupting counts and aggregates.

### 📖 Description

When a submission contains `Infinity` or `NaN` for a decimal field, two
problems occur:

1. **Invalid JSON response (this PR):** The backend statistics
computation (mean, median, stdev, mode) can produce non-finite float
values. These are not valid JSON, causing `JSON.parse` in the browser to
fail and the reports page to display an error. Fixed by introducing a
`SanitizedJSONRenderer` that recursively replaces any non-finite float
(`NaN`, `Infinity`, `-Infinity`) with `null` before serializing the HTTP
response.

2. **Incorrect statistics
([formpack#340](https://github.com/kobotoolbox/formpack/pull/340)):**
`NumField.parse_values()` was silently passing non-finite floats
through, causing them to be counted as valid provided responses and
corrupting `provided`, `total_count`, and aggregate statistics (mean,
median, etc.). Fixed upstream by raising a `ValueError` for non-finite
floats — the existing `_calculate_stats` error handler already treats
these as `not_provided`. This PR bumps the formpack dependency to
include that fix.

7284 of 11424 branches covered (63.76%)

13 of 14 new or added lines in 2 files covered. (92.86%)

27987 of 34438 relevant lines covered (81.27%)

1.6 hits per line