• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

supabase / cli / 38736404777
67%
develop: 46%

Build:
Build:
LAST BUILD BRANCH: gh-readonly-queue/develop/pr-7380-74d4774766936b6e6141a58d061c3396a01c85c9
DEFAULT BRANCH: develop
Ran 15 Oct 2026 12:41PM UTC
Jobs 1
Files 79
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

15 Oct 2026 12:39PM UTC coverage: 66.737%. Remained the same
38736404777

push

github

web-flow
ci(repo): add flaky-test check workflow (#7058)

## Summary

**TL;DR:** A new `Flaky check` workflow runs one commit on many fresh
runners with retries off and reports every test that both passed and
failed. It runs nightly on `develop`, weekly for e2e, and on demand
through the `run-flaky-check` PR label, so PRs that fix flaky tests can
show the fix holds.

```mermaid
flowchart LR
  T["nightly develop / weekly e2e<br/>run-flaky-check label / dispatch"] --> P["plan: commit, suites, N runs"]
  P --> M["N fresh runners per suite<br/>retries off, JUnit output"]
  M --> R["report: flaky / always failing<br/>runs without results"]
  R --> O["artifact + job summary<br/>PR comment"]
```

### Why

Flaky tests surface one at a time in unrelated PRs and get retried away.
There is no way to ask "is this commit stable?" or to show that a
flaky-test fix actually holds. `stack-stress.yml` covers two stack
scenarios only.

### What changed

- `flaky-check.yml` runs:
  - nightly: unit + integration ×10 on `develop`
  - weekly: CLI e2e (4 shards + `cli-e2e`) and stack whole-stack e2e ×3
- `run-flaky-check` label: unit + integration ×10 on the PR merge
commit, plus a `focused` suite that repeats the tests the PR affects
(`vitest --changed` against the merge base, `--repeats=4`, ×10 runs)
- manual dispatch: any ref, suites, run count, repeats, and file filter
- The label is removed when the run starts, so re-adding it starts
another run.
- A report job classifies each test as flaky (mixed results) or always
failing. It also lists runs that produced no usable results: timeouts,
crashes, missing JUnit reports, or missing uploads. Each failed
`--repeats` execution counts separately.
- The report goes to the job summary, a `flaky-check-report` artifact
(`report.md`, `report.json`), and one sticky PR comment. The report job
fails when it finds anything.
- `.github/MAINTAINERS.md` documents the triggers and the `focused`
caveats.

### Linked issue

None; maintainer change. Th... (continued)

4097 of 6139 relevant lines covered (66.74%)

7.37 hits per line

Jobs
ID Job ID Ran Files Coverage
1 38736404777.1 15 Oct 2026 12:41PM UTC 79
66.74
GitHub Action Run
Source Files on build 38736404777
  • Tree
  • List 79
  • Changed 0
  • Source Changed 0
  • Coverage Changed 0
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #38736404777
  • 35ee8bc2 on github
  • Prev Build on gh-readonly-queue/develop/pr-7179-a98214004972dc5bb76a05b12a0fd2ed5e194da3 (#38727207341)
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc