dnsimple / erldns / 21874370281

87%

Build Type

push

github

Committed by web-flow

Commit Message

Add pipeline suspension for non-blocking async operations (#327)

* Add pipeline suspension for non-blocking async operations

Introduce a suspension mechanism that allows pipeline stages to perform
blocking operations without stalling worker pools. This enables features
like external lookups to other nameservers or even databases, in a
non-blocking manner.

Architecture:

Pipeline stages can now return `{suspend, Msg, Opts, AsyncFun}` to pause
execution. The `AsyncFun` captures the blocking work to perform and returns
a standard pipeline return type when complete.

Transport handling differs by design:
- UDP: Suspensions hand off to `erldns_async_pool`, worker returns immediately
- TCP: Executes synchronously (dedicated per-request process)

The async pool is a bounded wpool (~4x schedulers) with CoDel queue
management to drop stale requests under load. Workers execute the blocking
`AsyncFun`, then send the result back to the original requester worker which
resumes the remaining pipeline.

Key changes:

- `erldns_pipeline`: New `{suspend, Msg, Opts, AsyncFun}` return type
  with continuation record for capturing execution state. Async work has
  a 3-deep chain limit to prevent infinite suspension loops. The module
  documentation for `erldns_pipeline` becomes now sufficiently
  complicated, and therefore it's moved to a separate markdown file at
  `src/pipes/erldns_pipeline.md` and integrated into the module.

- `erldns_async_pool`: New bounded worker pool using wpool. Receives
  continuations from UDP workers, executes blocking work, returns
  results via `gen_server:cast/2`. CoDel prevents queue buildup during
  overload.

- `erldns_proto_udp`: Handles `{suspend, Continuation}` by casting to
  async pool and returning immediately. Receives `{async_done,
  Continuation}` to resume pipeline and send response.

- `erldns_proto_tcp_request`: Handles `{suspend, Continuation}` by
  calling resume_pipeline synchronously - blocking is acceptable in
  dedic... (continued)

Run Details

111 of 146 new or added lines in 7 files covered. (76.03%)

1 existing line in 1 file now uncovered.

1690 of 1938 relevant lines covered (87.2%)

1574.38 hits per line