aio-libs / multidict / 36409791975

87%

push

github

Run Callgrind children from a fixed staging layout (#1623)

<!-- Thank you for your contribution! -->

## What do these changes do?

`benchmarks/callgrind_driver.py` gave different instruction counts for
identical code depending on where the checkout lived. Every path the
interpreter allocates at startup, and the listing of every directory it
imports from, shifts the heap the measured loop runs on. An allocating
operation such as an `istr` lookup (it copies its key) then takes
pymalloc's pool-full/pool-used transition on every iteration in one tree
and never in the other: identical code measured 700.5 vs 721.5 Ir on
`getitem_istr` from two trees whose paths differ by one character. That
is what made the #1619 measurement look like a +127 Ir `ctor_items`
regression.

The children now run under `-S` from a fixed-length temporary directory
holding a link to the base interpreter, a copy of their own scripts, and
one import directory with a link to each module they actually import
(for `multidict`, only its imported files, so stray build products do
not count either). They also get a private bytecode cache and a reduced
environment. The result records what was staged under `metadata.stage`.

While proving that, the free-threaded build turned out to be
nondeterministic between two runs of one tree. mimalloc seeds its arena
address from `getrandom()` and purges on a timer.
`MIMALLOC_ARENA_RESERVE=2GiB` (a reservation mimalloc leaves the kernel
to place) and `MIMALLOC_PURGE_DELAY=-1` pin both.

The driver also refuses to run without the Valgrind client requests (no
`pytest-codspeed`), whose whole-process counts cannot be compared with
bracketed ones, unless given `--whole-process`. Both traps are
documented in `docs/benchmark.rst` and AGENTS.md.

## Are there changes in behavior for the user?

No; this only touches the benchmark tooling. Contributors now need
`pytest-codspeed` installed (or `--whole-process`) to run the Callgrind
driver. Absolute counts shift... (continued)

784 of 1568 branches covered (50.0%)

Branch coverage included in aggregate %.

8613 of 9234 relevant lines covered (93.27%)

1.87 hits per line