aio-libs / multidict / 36467795488

87%

push

github

Keep update()'s doomed entries whole until the batch ends (#1628)

<!-- Thank you for your contribution! -->

## What do these changes do?

`update()` half-deleted the later matches of a key as it went (key and
value set to `NULL`, identity kept) until `md_post_update()` finished
them off, and kept its marks keyed by entry index, dropping them
whenever anything else touched the table. Python code that runs between
the items of one `update()` (the argument's own iterator, a key's
`lower()`, a finalizer of a dropped item) could therefore:

- crash reading a half-deleted entry through `get()`, `getall()`,
`items()`, `copy()` or `popall()`;
- make a later item for the same key overwrite a value the call had just
written, since any mutation dropped the marks;
- have `merge()` skip a key a pending deletion was about to remove.

Doomed entries now stay whole until `md_post_update()` removes them,
marked only in the batch's bitmap. Each is recorded with the value it
held when doomed, referenced so the address cannot be reused, and is
removed only if it still holds it: a nested `update()` or `md[key] =
value` run between items that writes to it wins. (Only the value is
compared, since reading a `CIMultiDict` key swaps the stored `str` for
its `istr`.) The marks stay valid because nothing reuses an entry index
while a batch is in flight: `md` counts its batches in `is_ci`'s padding
(the object stays 64 bytes), and while there are any a resize keeps
every entry at its index, `popitem()` leaves its trailing holes, and a
clear or re-init starts the new table with a hole per old entry. A
generation bumped with every new table tells a batch to widen its
bitmaps. The remap, the sweep and the half-deletion all go.

The pure-Python implementation had the same shape with `HASH_MARK` and
`key=None` in the table, and returned `None` for doomed pairs; it now
keeps its marks in dicts keyed by entry id, and holds replaced pairs
until the call ends, as C does.

## Are there c... (continued)

790 of 1580 branches covered (50.0%)

Branch coverage included in aggregate %.

147 of 153 new or added lines in 2 files covered. (96.08%)

8777 of 9417 relevant lines covered (93.2%)

1.86 hits per line