hyperledger / fabric-x-committer / 30811450871

91%

main: 91%

Build Type

Pull #736

github

Committed by liran-funaro

Commit Message

[coordinator, sidecar, vc] Report queue sizes on scrape

#### Type of change

- Bug fix
- Improvement (improvement to code, performance, etc)

#### Description

- Replace the three remaining ticker-based queue monitors with gauges evaluated on scrape, and
  delete the sampler goroutines. `service/sidecar/sidecar.go`'s `monitorQueues` was never called
  after #544 removed its call site, so `sidecar_relay_input_block_queue_size` and
  `sidecar_relay_output_committed_block_queue_size` reported a flat zero; both work again.
- Add `monitoring.NewChannelLenGauge` for a channel that lives as long as the service, and
  `NewAtomicChannelLenGauge` for one that is replaced at runtime. A sidecar session's queues and
  each manager's `txBatchQueue` are reported through an `atomic.Pointer` the owner installs, so a
  scrape between sessions reads zero rather than the previous session's last value. Recreating those
  queues is how stale entries are dropped, so promoting them to a plain field was not an option.
- Add gauges for ten queues that had none: `vcservice_batcher_input_queue_size`,
  `sidecar_relay_mapped_block_queue_size`, `sidecar_relay_input_status_batch_queue_size`, the four
  `sidecar_notifier_*` queues, `coordinator_{verifier,vcservice}_pending_batch_queue_size`, and
  `coordinator_grpc_in_progress_transaction` — the counter the
  `numTxsInProgress >= readyCount >= 0` invariant is written against.
- Remove the `queue-monitor-sampling-time` config key: the dependency graph's sampler was its only
  reader. Deployed YAML that still sets it keeps loading, as `v.Unmarshal` ignores unknown keys.
- Gather each service's pipeline channels into a `queues` struct built before the metrics that
  observe them, so a queue gauge cannot be registered against a channel that does not exist yet.
- Teach `scripts/metrics_doc_extract.py` the helper call shape (opts as the second argument) and
  identifiers declared as string constants. Without the second change, hoisting a... (continued)

Pull Request Pull Request #736: [coordinator, sidecar, vc] Report queue sizes on scrape

Coverage Stats

187 of 188 new or added lines in 14 files covered. (99.47%)

13 existing lines in 6 files now uncovered.

10665 of 11683 relevant lines covered (91.29%)

33042.21 hits per line