aio-libs / multidict / 35901354185

87%

push

github

Publish dict versus multidict comparison tables (#1532)

<!-- Thank you for your contribution! -->

## What do these changes do?

`docs/benchmark.rst` explained how to run the benchmarks but published
no numbers, so the question a reader arrives with, what `MultiDict`
costs compared with a plain `dict`, went unanswered, and the
free-threaded build's overhead was undocumented. The page now carries
per-operation tables for `dict`, `MultiDict` and `CIMultiDict` on the
GIL build, the same on the free-threaded build, a free-threading
overhead summary with `dict` as the control, and a pure-Python backend
table. Fifteen rows, one per operation the two types share.

The numbers are Callgrind instruction counts rather than wall clock,
because they do not depend on scheduling or CPU frequency and so
reproduce on a busy machine; a regression shows up as a changed integer.
`benchmarks/callgrind_driver.py` measures each cell with four child
processes, two round counts per arm. Differencing two non-zero round
counts cancels interpreter startup, and subtracting an arm with the
operation elided cancels the driving loop, which is worth a few hundred
instructions per iteration and would otherwise compress every ratio
toward 1. `benchmarks/render_tables.py` turns two runs into the tables
and rewrites the block between markers in the page.

Getting there needed the scripts restructured. The pyperf script built
each benchmark out of `exec`'d source templates, which left nowhere to
run setup between measured iterations and limited it to six
non-destructive operations. `benchmarks/operations.py` now holds every
operation as a setup/run/noop triple that all three entry points read,
and the rebuild a destructive operation needs happens in setup, outside
the measured region, so `pop()`, `clear()` and `del d[key]` can be
measured without counting it.

Both backends are covered: the tables carry `MultiDict` and
`CIMultiDict` for the C extension and for the pure-Python
implementa... (continued)

765 of 1530 branches covered (50.0%)

Branch coverage included in aggregate %.

7221 of 7604 relevant lines covered (94.96%)

1.9 hits per line