evolution-gaming / kafka-flow / 40857424675

86%

push

github

Unload a key only after its state was persisted (#934)

* Unload a key only after its state was persisted

`persistPeriodicallyAndUnloadOrphaned` removed a key whenever the unload
threshold was crossed, whether or not the persist that preceded it went
through. With `ignorePersistErrors` on, a failed persist is logged and
the key is unloaded anyway; `KeyContext.remove` also drops the offset it
held, so the partition can commit past state that never reached the
store and the next recovery skips those events.

Gate the unload on the persist outcome: an unpersisted key stays loaded
and keeps holding its offset until a later tick persists it.

* Log an ignored persist failure at WARN

In `persistPeriodicallyAndUnloadOrphaned` a key whose persist failed
stays loaded and retries every tick, logging this line each time, so
its rate shows how many keys are stuck in memory. Log it at WARN, as
AdditionalStatePersist does for the failures it ignores.

* Tidy the ignorePersistErrors docs

Reflow the scaladoc after the WARN rewording, and note in the
overview that a key kept after an ignored persist failure stays in
memory.

* Test that a kept key is unloaded once a later persist succeeds

A key kept loaded after an ignored persist failure must not stay in
memory for good once the store is back.

* Persist the kept key at a higher offset in the recovery test

A key whose persist failed, e.g. a snapshot too big for the store, can
succeed later once its state has moved on.

392 of 463 branches covered (84.67%)

Branch coverage included in aggregate %.

1901 of 2266 relevant lines covered (83.89%)

1.31 hits per line