lsm / HyperNeo / 30736748571

83%

push

github

Instrument pending external-event queue health (#2257)

* feat: instrument pending external-event queue health

Add an in-memory metrics collector for the pending external-event delivery
queue, plus a delivery-terminal hook on ExternalEventStore so every delivered
and terminal-failed transition is counted once from a single observation
point. SpaceRuntime records enqueues (by source + target run/node state),
flush attempts, claim conflicts, stale-session skips, and structured logs at
the previously-silent skip seams, and exposes getQueueHealthSnapshot() merging
cumulative counters with live gauges (depth, event-age p95, in-flight, digest
backlog, persisted pending). Surface it via the space.externalEvents.queueHealth
RPC. Service wires the shared metrics to the store hook.

* feat(web): add pending external-event queue-health operator view

Add a QueueHealthSummary panel to the external-events settings that fetches
the daemon-wide queue-health snapshot via the new space.externalEvents.queueHealth
RPC and renders cumulative counters (enqueue by source/target state, flush,
delivered, failures by reason/category, skips) and live gauges (depth, age
p95, in-flight, digest backlog). Add typed store helper + QueueHealthSnapshot
types.

* fix: address queue-health review (perf, skips, category mapping)

Address PR #2257 review feedback:

- Compute persisted-pending health via SQL aggregates
  (summarizePendingDeliveries: COUNT/MIN/MAX/AVG + LIMIT/OFFSET p95) instead of
  materializing every pending row + a full in-memory sort on each snapshot read.
- Count paused-space deferrals as a separate pausedSpaceSkips counter (recorded
  at the digest-requeue, fresh-delivery, and retry-injection pause sites), so
  staleSessionSkips stays a true session-loss signal and the pause-induced
  backlog scenario is no longer reported as zero skips.
- Instrument preservePendingDigestItem with recordEnqueue so rehoused digest
  items keep the cumulative enqueue counter consiste... (continued)

12015 of 17172 branches covered (69.97%)

Branch coverage included in aggregate %.

313 of 325 new or added lines in 9 files covered. (96.31%)

102595 of 120644 relevant lines covered (85.04%)

300.44 hits per line