evolution-gaming / kafka-flow / 40864847937

86%

push

github

Count persist failures in the database metrics (#975)

* Count persist failures in the database metrics

With ignorePersistErrors a failed persist is only logged, so a store
that keeps failing is invisible in the metrics. The snapshot, journal
and key database metrics now count every failed persist, ignored or
not, as *_database_persist_failures, for any backend they wrap,
including the Kafka persistence's snapshot writes when it is given a
FlowMetrics.

Counting a failure needs MonadThrow instead of Monad in these metrics,
PersistenceModuleMetrics and FlowMetrics.of. That is binary
incompatible: metrics sets versionPolicyIntention to Compatibility.None
and the next release has to be a major.

* Count only failures of the database write itself

* Explain which persist-failure counter moves

* Say which persist-failure counter moves more briefly

* Name the persist-failure counters with a _total suffix

Prometheus counters should end in _total. simpleclient 0.9, upstream or
the evo fork, exposes a counter under its name as is, so without the
suffix these counters came out unconventionally named. prometheus-metrics
1.x accepts a name ending in _total and exposes it with a single _total,
so the suffix is right on both backends.

393 of 466 branches covered (84.33%)

Branch coverage included in aggregate %.

2009 of 2331 relevant lines covered (86.19%)

1.35 hits per line