aio-libs / multidict / 36402130758

87%

push

github

Stress re-init with finalizers against lock-free readers (#1621)

<!-- Thank you for your contribution! -->

## What do these changes do?

Adds a threaded stress test for the case #1619 left uncovered: one
thread re-initializes a shared `MultiDict` / `CIMultiDict`, alternating
`d.__init__(pairs)` (`md_init()` then extend) and
`d.__init__(same_kind_md)` (`md_clone_from_ht()`), with values whose
`__del__` reads the multidict and adds to it. Four other threads run
lock-free `get()`, `getone()`, `__getitem__`, `__contains__` and
`len()`, iterate `items()` / `keys()` / `values()`, and take `copy()`
snapshots. On free-threaded builds almost every finalizer runs in a
reader's drain, not the writer. Every snapshot must hold a prefix of one
generation plus `"late"` tokens, and after two `clear()` calls every
token must be freed.

Under TSan the test found two data races, both fixed here:

- `md_clone_from_ht()` did one `memcpy()` of the whole source table,
header included, so it read `num_readers` while lock-free readers of the
source updated it atomically. On free-threaded builds the copy now skips
the reader fields and zeroes them; GIL builds keep the single
`memcpy()`.
- `_md_install_keys()` stored `md->is_ci` on every re-init while
lock-free lookups read it in `md_calc_identity()`. It now stores only
when the value changes. A re-init through the type's own `__init__()`
never changes it.

Neither race was seen to crash. The first reads a value that is thrown
away and the second writes the value already there, but both are
undefined behaviour, and CI's TSan job (`halt_on_error=1`) would fail on
this test without the fixes.

## Are there changes in behavior for the user?

No.

## Is it a substantial burden for the maintainers to support this?

No. The test runs in about 0.5s on a free-threaded release build.

## Related issue number

Follow-up to #1619.

## Checklist

- [x] I think the code is well written
- [x] Unit tests for the changes exist
- [x] Document... (continued)

784 of 1568 branches covered (50.0%)

Branch coverage included in aggregate %.

59 of 81 new or added lines in 1 file covered. (72.84%)

8611 of 9226 relevant lines covered (93.33%)

1.87 hits per line