cybertec-postgresql / pgwatch / 25668650925

86%

Build Type

push

github

Committed by pashagolub

Commit Message

[!] reaper goroutine consolidation via `pgx.Batch` queries (#1316)

The improvements stem from consolidating N per-metric goroutines into a single per-source SourceReaper that uses a GCD-based (Greatest Common Divisor) tick loop and pgx.Batch for batched SQL execution, dramatically reducing goroutine overhead, connection contention, scheduling pressure, and memory allocation churn.

For 6 sources test run batch implementation eliminates 414 per-metric goroutines, replacing them with just 6 per-source goroutines. All infrastructure goroutines (pool health checks, sinks, logging, web server) remain identical between both implementations.

The 4× reduction in stack memory directly reflects the goroutine count difference — each goroutine requires its own stack.

At steady state (after initial warmup), the batch implementation allocates approximately 8–9× less memory per minute.

While GC cycle counts are similar, the CPU fraction spent on GC is ~3× lower for batch because each cycle processes far fewer live objects (45K vs 510K).

---------

Co-authored-by: 0xgouda <gouda0x@gmail.com>
Co-authored-by: Mazen Kamal <71020170+Mazen050@users.noreply.github.com>

Coverage Stats

222 of 287 new or added lines in 6 files covered. (77.35%)

6 existing lines in 2 files now uncovered.

4653 of 5434 relevant lines covered (85.63%)

0.98 hits per line