kobotoolbox / kpi / 35956905562

83%

master: 76%

Build Type

push

github

Committed by web-flow

Commit Message

feat(openapiValidator): add OpenAPI request/response validation middleware DEV-1740 (#7314)

### 📣 Summary

The API and its published documentation are now checked against each
other automatically, so the API documentation stays accurate.

### 📖 Description

KoboToolbox publishes a description of its API that integrations and
third-party tools rely on. Until now nothing verified that the API
really matches it, so the two could drift apart unnoticed. Requests and
responses are now compared against that description automatically and
any difference is reported to the team, so it gets fixed instead of
reaching the documentation. Nothing changes in how the product behaves
today.

### 💭 Notes

Proof of concept for DEV-1740, picked up from Olivier's branch (~700
commits behind main at the time).

- New app `kobo.apps.openapi_validator`: middleware validating requests
and responses against `static/openapi/schema_v2.json`.
- Test settings only. `OPENAPI_VALIDATION` is hard-coded off in
`base.py` and on in `testing.py`, and the middleware refuses to load
unless `settings.TESTING` is true (`MiddlewareNotUsed`, so Django drops
it from the stack): a deployed environment never runs it whatever the
flags say and pays nothing. Enabling it on live traffic is DEV-1740's
follow-up, not this PR.
- Strict in test settings: a mismatch fails the test that triggered it
unless listed in `OPENAPI_KNOWN_MISMATCHES` — 85 `(error_code, route,
method)` entries, each one a real schema bug to fix and delete.
- Entries are keyed by endpoint, not by test, so no test file is
touched: the 1189 per-test entries the suite produces are only 85
distinct bugs, and endpoint keys survive test renames. Trade-offs: a new
test hitting an already-listed endpoint passes silently, and the key
ignores the status code, so a listed `response-validation` silences
drift on every status that route returns. Adding the status to the key
needs a full-suite regeneration — follow-up.
- Request-side errors are ... (continued)

Coverage Stats

10341 of 13696 branches covered (75.5%)

168 of 223 new or added lines in 7 files covered. (75.34%)

2964 existing lines in 78 files now uncovered.

33002 of 39788 relevant lines covered (82.94%)

5.51 hits per line