ben-manes / caffeine / #2682

99%

push

github-actions

Protect reference caches from a discarding executor

The maintenance task is scheduled on an executor and disables further
scheduling until the submitted task runs. If the task is not run when
expected this could cause a memory leak or block writers due to
backpressure. This is detected when the write buffer is full, which
causes writers to provide assistance by barging the eviction lock and
performing the maintenance work themselves.

The write buffer was only enabled for the size or expiration policies
as their data structures require replaying activity. A weak or soft
reference cache disabled this as its data structures are managed by
the JVM. Instead that policy leveraged the read buffer as a cheap way
to trigger periodic maintenance which polls the Java ReferenceQueue.
This assumed that there were no failure scenarios that could halt
scheduling.

Unfortunately this was overly optimistic. If the executor silently
discards tasks then an out-of-memory error could occur because the
cache would not recover and restart eviction. This could happen if
using a discarding policy (such as the RejectedExecutionHandler
implementations offered by ThreadPoolExecutor, by ExecutorService's
shutdownNow(), or a bug in the executor. ForkJoinPool suffered from
missed submissions in JDK-8078490 (fixed in jdk 8u60) and recently
discovered in jdk 17 (see newrelic/newrelic-java-agent#558).

The cache should now recover and continue to evict in cases where
the maintenance task is not run promptly. Note that an executor
that discards tasks is inherently broken and can cause asynchronous
loads to never complete (such as refreshAfterWrite or AsyncCache).
In those cases it is left to the user to recover, for example by
using CompletableFuture.orTimeout(duration) with the computation.

12 of 12 new or added lines in 1 file covered. (100.0%)

6372 of 6784 relevant lines covered (93.93%)

0.94 hits per line