safe-global / safe-events-service / 40663250356

88%

push

github

fix: Close RabbitMQ connection cleanly on shutdown (#639)

* fix: Close RabbitMQ connection cleanly on shutdown

Shutdown hooks were not enabled, so there was no SIGTERM listener.
Node is PID 1 in the container and ignored SIGTERM, kept consuming, and
was killed by SIGKILL at the end of the grace period. RabbitMQ logged
"client unexpectedly closed TCP connection".

- Enable shutdown hooks and force close open HTTP connections (SSE),
  so shutdown can finish
- Exit with process.exit() after the hooks. Node is PID 1 in the
  container, and the signal Nest sends again to itself is ignored, so
  the process stayed alive until SIGKILL if the event loop was not empty
- On shutdown, cancel the consumer, wait for in-flight messages
  (AMQP_SHUTDOWN_TIMEOUT, default 15s) and close the connection
  (bounded to 5s). Messages not acked in time are requeued by RabbitMQ
- Readiness fails while shutting down, without logging an error, and
  no new connection is opened
- Close the webhook agent in onApplicationShutdown, after the wait
- Move parseNumberConfig to common utils to reuse it

Refs PLA-2146

* refactor: Rename parseNumberConfig fallback to defaultValue

Same name as in ConfigService.get(KEY, default) and in the warning log.

* test: Add parseNumberConfig unit tests

Cover valid values, a value equal to min, unset values, invalid values
and a value below min.

* fix: Requeue messages and cancel consumers registered after shutdown started

NestApplication.init() does not set initializationPromise, so on SIGTERM
the shutdown hooks can run while onApplicationBootstrap is still
subscribing. A consumer registered after cancelAll() was never
cancelled, and its messages were processed without being waited for.

- Cancel the consumer if shutdown started when consume() resolves
- Requeue (nack) any message delivered after shutdown started instead
  of processing it

312 of 367 branches covered (85.01%)

Branch coverage included in aggregate %.

57 of 61 new or added lines in 6 files covered. (93.44%)

733 of 829 relevant lines covered (88.42%)

21.76 hits per line