supabase / cli / 32829565108
62%
develop: 62%

LAST BUILD BRANCH: dependabot/docker/apps/cli-go/pkg/config/templates/docker-minor-e722515170
DEFAULT BRANCH: develop
Ran
Jobs 1
Files 95
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 62.406%. Remained the same
32829565108

push

github

web-flow 
fix(cli): bound piped stdin buffering (CLI-2223) (#6290)

## TL;DR

fixes the CLI being OOM killed when a long running command is fed by an
unbounded pipe such as `yes | supabase db push` which was caused by the
line reader holding stdin open but idle between prompts
and is now fixed by reading piped stdin ahead into a bounded queue
instead of on demand....

## what's biting the user?

Go read stdin on demand with one `bufio.Scanner`, which was safe because
the pipe pushed back and `yes` simply blocked. The port kept that shape,
but Bun's `process.stdin` cannot be throttled:
once a reader is attached Bun keeps reading the pipe, and `pause`,
`destroy`, `unref` and detaching the listener all fail to stop it.
An idle reader therefore buffers for the whole migration, reaching 5 to
6 GB RSS in seconds. `--yes` escapes it only because the reader is lazy
and never opens stdin....

## fixed now by:

Piped stdin is drained continuously into a bounded queue of 1024 lines,
which holds memory flat at around 120-140MB Overflow drops newest first,
so the lines prompts read stay in pipe order.
A TTY keeps the on demand reader, since a keyboard cannot outrun the
prompts and a drainer would take keystrokes a later prompt needs.....

> [!NOTE]
> The line cap and the drop on overflow are deliberate, not an oversight
a source that honors backpressure needs no cap at all
but it replaces the platform `Stdio` service rather than one consumer of
it, so it is better to be tracked separately.

<details>
<summary>alt approach considered: backpressured fd 0 (maybe a later
followup) </summary>

Bun's `process.stdin` cannot be throttled, but `fs.createReadStream("",
{ fd: 0 })` can.
Measured with the same reader logic and the same `yes |` producer:

| source | RSS under `yes \|`, 8s idle |
| --- | --- |
| `process.stdin`, on demand (before) | 5 to 6 GB, climbing |
| `process.stdin`, bounded queue (this PR) | ~150 MB |
| `fs.createReadStream("", { fd: 0 })` | ~100 MB |

With real b... (continued)

5571 of 8927 relevant lines covered (62.41%)

6.65 hits per line

Jobs
ID Job ID Ran Files Coverage
1 32829565108.1 95
62.41
 GitHub Action Run
Source Files on build 32829565108