supabase / etl
81%

DEFAULT BRANCH: main
Repo Added
Files 250
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH main
branch: SELECT
CHANGE BRANCH
x

coverage: 80.582% (+0.02%) from 80.562%
41674793996

push

github

web-flow 
fix(clickhouse): increase leniency of retries on proxy 5xx errors and startup probe (#1200)

* fix(clickhouse): retry proxy 5xx error bodies

The clickhouse crate keeps the HTTP status only when an error body is
empty. A proxy that answers with its own body, such as a bare
"Bad Gateway" or an nginx error page, therefore produced a BadResponse
with neither a ClickHouse code nor a status. It was classified as
permanent and stopped replication for manual retry.

Recognize the server errors proxies commonly report in their own body,
as a bare reason phrase or a status line in an error page, and treat
them like the bare 5xx status. Redaction keeps that status, so errors
read "502 Response body omitted" instead of an opaque
"Response body omitted".

* fix(clickhouse): retry startup version probe

The engine support check runs one SELECT version() at startup under the
12 second connectivity budget. A timeout or proxy 5xx failed startup at
once, and replicators retried only through pod restarts. Prod saw this
across several pipelines in the same minutes.

Retry retryable probe failures with the shared backoff helper: four
retries with 2, 4, 8, and 16 second delays. A destination that never
answers now fails startup after up to about 90 seconds. Permanent
failures, such as a rejected login, still fail on the first attempt.
A shutdown signal still cancels startup, including the backoff sleep.

Integration tests put a proxy in front of the test ClickHouse: startup
recovers after a 502 body and fails at once on a 403 body.

34 of 35 new or added lines in 2 files covered. (97.14%)

16 existing lines in 5 files now uncovered.

60758 of 75399 relevant lines covered (80.58%)

29960.4 hits per line

Relevant lines Covered
75399 RELEVANT LINES 60758 COVERED LINES
29960.4 HITS PER LINE
Source Files on main

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
41674793996 main fix(clickhouse): increase leniency of retries on proxy 5xx errors and startup probe (#1200) * fix(clickhouse): retry proxy 5xx error bodies The clickhouse crate keeps the HTTP status only when an error body is empty. A proxy that answers with it... push web-flow github
80.58
41671116447 codex/string-pipeline-id-cutover Merge 428741419 into 3aff53f1d Pull #1198 web-flow github
80.49
41670980870 jm/clickhouse-proxy-5xx-retry Merge 341d6c09a into 3aff53f1d Pull #1200 web-flow github
80.54
41667499999 fix/snowflake-recovery-barrier Merge 72fa2b1dc into 3aff53f1d Pull #1199 web-flow github
80.66
41650142572 fix/identity-parent-fallback-partitions-only Merge ffa1dff13 into 3aff53f1d Pull #1117 web-flow github
79.56
41650679573 bnjjj/ducklake-catalog-pooler Merge e4e885925 into 3aff53f1d Pull #1197 web-flow github
80.92
41633848208 main feat(migration): Backfill pipeline IDs and prepare string keys (#1194) push web-flow github
80.56
41626089456 main refactor(config): derive secret string deserialization (#1196) push web-flow github
80.56
41625723959 bnjjj/ducklake-catalog-pooler Merge 204d39903 into f9928aa13 Pull #1197 web-flow github
80.94
41619852410 backfill-ids Merge 157ef32a7 into f9928aa13 Pull #1194 web-flow github
80.56
See All Builds (3395)