aio-libs / multidict / 36311162759

86%

push

github

Add watched-mode and FT benchmarks, split the benchmark job in two (#1608)

<!-- Thank you for your contribution! -->

## What do these changes do?

Adds `tests/test_watched_benchmarks.py`, which runs every mutation
benchmark shape from `test_multidict_benchmarks.py` (insert, add,
setitem, pop, popitem, delitem, clear, update, extend, merge) on a
multidict that has a no-op watcher attached, for both `MultiDict` and
`CIMultiDict`. Nothing had been measuring the watched path before this.
Views, iterators and reads are left out on purpose.

Watchers can only be reached through the C API, so the file uses
`multidict._testcapi` and is skipped on the pure-Python build. It adds
two harness helpers: `md_add_noop_watcher()`, whose callback does
nothing, so the numbers are not dominated by `record_event()`, and
`md_watch_noop()`, which watches without appending `user_data` to the
keep-alive list. That matters because a `copy()` is not watched, so
every copy is watched inside the timed loop. The helpers exist only for
benchmarking, so they are not mirrored into `_testcyapi.pyx`.

The CodSpeed job is also split into two parallel jobs of similar length.
The new `benchmark_shard_2` mark goes on the bulk
`update()`/`extend()`/`merge()` and removal benchmarks in
`test_multidict_benchmarks.py` and on the whole views file, and each job
runs all three benchmark files, selecting with `-m`. According to the
per-test timings from the previous CI run on this PR, that gives about
330 s of measurement for one job and 350 s for the other. On master, the
single job takes about 9 minutes. An earlier three-way split by file did
not help, because the multidict file alone took 10.5 minutes.

The FT build is now benchmarked too, with the same two jobs on 3.14t,
which makes four jobs in all. CodSpeed identifies a result by the pytest
node ID alone and flags a benchmark that is reported twice in one run,
so on an FT interpreter `conftest.py` adds an `ft` part to every
benchmark's para... (continued)

781 of 1562 branches covered (50.0%)

Branch coverage included in aggregate %.

159 of 172 new or added lines in 5 files covered. (92.44%)

8423 of 8992 relevant lines covered (93.67%)

1.87 hits per line