aio-libs / multidict / 35499797092

87%

push

github

Fetch list items as new references on free-threaded builds (#1437)

## What do these changes do?

Bulk updates from a list and `[key, value] in md.items()` borrowed items
from a list the caller still owns and increfed them in a separate step.
On a free-threaded build another thread can drop the item in between, or
shrink the list between its length check and the reads, and the C
extension reads freed memory.

Each item is now taken as a new reference with `PyList_GetItemRef` on
free-threaded builds, which fetches atomically and locks the list only
if its lock-free attempt fails; GIL builds keep the `PyList_GET_ITEM`
macro, so their hot paths are unchanged. Follows #1379, which fixed the
stale length in the same loop.

A critical section on the list, as #1433 takes on a dict argument, would
not do: it covers neither a `[key, value]` item inside the list nor the
probe handed to `in`, and it is released whenever the thread detaches,
which a key's `__hash__` or `__eq__` can cause mid-loop. A strong
reference per fetch is what actually holds.

## Are there changes in behavior for the user?

On free-threaded builds only: a call that finds a list item gone after
the list's length was checked now raises `RuntimeError("list changed
size during iteration")`, from `update()`, a `[key, value]` item and
`[key, value] in md.items()` alike. A list that is simply shorter by the
next iteration of `update()` still ends the update there, as before
(#1379) and as in a plain `for` loop.

## Is it a substantial burden for the maintainers to support this?

No. One helper in `hashtable.h`, used at three sites.

## Related issue number

None.

## Checklist

- [x] I think the code is well written
- [x] Unit tests for the changes exist
- [ ] Documentation reflects the changes -- N/A, no API change
- [ ] If you provide code modification, please add yourself to
`CONTRIBUTORS.txt` -- N/A, no such file
- [x] Add a new news fragment into the `CHANGES/` folder

---------

Co-authored... (continued)

762 of 1524 branches covered (50.0%)

Branch coverage included in aggregate %.

83 of 83 new or added lines in 1 file covered. (100.0%)

6320 of 6588 relevant lines covered (95.93%)

1.92 hits per line