kubernetes-sigs / inference-perf / 39822622994
84%

DEFAULT BRANCH: main
Ran
Jobs 1
Files 136
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 84.288% (+0.1%) from 84.139%
39822622994

push

github

web-flow 
test: cover the Prometheus query execution and collection flows (#761)

Fixes #660. Part of #792, Test backfill.

#660 asked for three pins on the Prometheus client: golden PromQL from
the query builder, `execute_query` against canned HTTP responses, and
the `collect_metrics_*` flows against canned query results. #382
delivered the goldens, and main has since gained `execute_query` tests
in `test_prometheus_client.py`. Measured on main,
`tests/required/client/metricsclient` leaves `base.py` at 79%. The
missing lines are the constructor guards, `wait()`, both
`collect_metrics_*` bodies and the None-response guard (34, 44, 51-52,
64-72, 87-103, 185-186).

This adds `test_prometheus_collection.py` and takes `base.py` to 100%.
`execute_query` returns None on every failure it handles, so a failed
query reaches the report as missing rather than as a real-looking zero.
Each of those paths is pinned with its error log. The empty result set
is pinned as the one silent None, since the collector retries None with
the underscore-escaped name. Two bad responses still raise out of report
generation: a 200 whose body is not JSON, and a JSON body that is not an
object. They are pinned in their own class as current behaviour, not
endorsed.

The collection window comes from the stage's own timestamps, `end_time +
scrape_interval + PROMETHEUS_SCRAPE_BUFFER_SEC`. A stage running 50.0 to
100.0 at a 30s interval evaluates at 132.0 over an 82.0 window. Wrong
arithmetic there reads another stage's samples with nothing to flag it.
Two consequences are pinned as documented, not endorsed. A stage
starting within one interval plus buffer of its predecessor's end lands
inside the predecessor's queried range. A FAILED stage is queried
exactly like a COMPLETED one.

Changes:
* `test_prometheus_collection.py`: `execute_query` parsing (including
"0" to 0.0), 6-decimal rounding, first series only, the request it
sends, every None path with its log line, the two raising paths and
NaN/+... (continued)

12462 of 14785 relevant lines covered (84.29%)

0.84 hits per line

Jobs
ID Job ID Ran Files Coverage
1 39822622994.1 136
84.29
GitHub Action Run
Source Files on build 39822622994