supabase / etl / 41404196493

81%

Build Type

push

github

Committed by web-flow

Commit Message

feat(xtask): Add CI metrics collector (#1170)

* feat(xtask): Add CI metrics collector

Add `cargo x ci-metrics migrate|sync`, which collects GitHub Actions
runs, jobs, benchmark manifests and code-scanning metadata into a
Postgres `ci` schema, with read-only `ci_report` views for Grafana.

Syncs are level-triggered: each run compares wanted per-day coverage
with stored state, so outages and crashes heal on the next sync. Writes
are idempotent upserts keyed on GitHub ids; jobs only move forward and
artifacts are untrusted, size-capped input.

Move the GitHub REST transport into a shared module so
`benchmark-compare` and the collector use one client.

* feat(xtask): Ingest benchmark backfill results

Benchmark backfills upload `benchmark-backfill-*` artifacts, which the
collector skipped, so backfilled history never reached the database.
Accept both prefixes; the manifest trigger tells the runs apart.

Parse the manifest commit time as RFC 3339 and store it in UTC, and
expose it on `ci_report.benchmark_metric` so backfilled points plot at
the measured commit's time.

* feat(xtask): Add CI report views for dashboards

Move logic the dashboard repeated in panel SQL into `ci_report` views:
`workflow` labels (CodeQL default setup and workflow share a name),
`flaky_job`, `pr_time_to_green`, and a `scheduled_daily` that matches
the health panel. Fold the benchmark commit-time column into the
initial migration, since neither migration has shipped.

Add a build script so edited migrations are re-embedded: Cargo does
not track `sqlx::migrate!` inputs on stable Rust.

* fix(xtask): Harden CI metrics sync per review

- Skip manifests whose attempt GitHub no longer returns instead of
  failing the foreign keys on every sync.
- Strip URLs from reqwest errors so signed artifact download URLs never
  reach logs.
- Treat a 403 from code scanning as unavailable rather than fatal, so
  the sync still records success.
- Mark analyses as not backfilled while a multi-page ... (continued)

Coverage Stats

952 of 2418 new or added lines in 11 files covered. (39.37%)

33 existing lines in 7 files now uncovered.

60383 of 75098 relevant lines covered (80.41%)

36051.77 hits per line