FlexMeasures / flexmeasures / 31199773705

84%

Build Type

push

github

Committed by web-flow

Commit Message

Feat: Sentry rate limiting (#2366)

* rate-limit how many events are sent to Sentry each day

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* do not emit error logs when the alembic revision is not on head, just warn

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* docs: update changelog entry  with pr number

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>

* style: run pre-commit

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>

* docs: clarify Sentry daily limit sampling interaction.

Document that sampled-out errors still count towards the daily rate limit.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>

* utils/sentry: bound Redis waits for daily rate limiting

Use a dedicated Redis connection with one-second connect and read timeouts and no retries, so Sentry reporting fails open without changing the Redis connection used by RQ jobs.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>

* tests/utils: cover bounded Sentry Redis failures

Verify the dedicated Sentry Redis connection has bounded waits and no retries, and that connection and timeout errors return the original event.

Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>

* utils/sentry: report the Alembic schema mismatch once a calendar day

Every FlexMeasures process logs this error while starting up, and hosts run CLI commands often, so reporting each start spent too much of the Sentry allowance.
Keep the message at ERROR, so the mismatch stays visible in the host's own logs and in Sentry, and let a log record name itself for deduplication through the `extra` argument instead.
Records carrying such a key are reported once per UTC calendar day per key, and the startup error keys itself on the current and expected revisions, so a mismatch against other revisions is reported right away.
Deduplication runs before the daily rate limiter, so suppressed reports do not count towards t... (continued)

Coverage Stats

96 of 98 new or added lines in 5 files covered. (97.96%)

87 existing lines in 1 file now uncovered.

17312 of 20582 relevant lines covered (84.11%)

0.84 hits per line