pyro-ppl / numpyro / 30701977069
92%

DEFAULT BRANCH: master
Ran
Jobs 3
Files 86
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 92.364%. Remained the same
30701977069

push

github

web-flow 
feat: compare branch speed against master and report it on the PR (#2233)

* feat: compare branch speed against master and report it on the PR

Adds a benchmark suite and the CI mechanism that measures a pull request
against its merge base, then posts the difference as a sticky comment.

benchmarks/ holds 32 benchmarks across four suites -- MCMC, SVI,
distributions and effect handlers. Each is a setup function returning a
zero-argument run callable, so data generation stays out of the
measurement. The harness clears the JAX caches, calls it once, then
several times warm: the fastest warm call is reported as run time, and
the excess of the cold call over it as compile time.

The workflow is opt-in behind a `run-benchmarks` label, because a full
comparison takes tens of minutes. It builds a venv for the head and one
for the merge base, pinning the base to the head's JAX version so the
report measures NumPyro rather than JAX, and measures both with the head
revision of benchmarks/ so a benchmark added by the PR still runs
against the base. Where it cannot run there, the row is reported as n/a
with the reason rather than dropped.

Commenting is split into a second workflow_run workflow: a pull_request
build of a fork gets a read-only token, so the benchmark job holds no
write permission and only uploads an artifact, and the privileged job
never executes code from the pull request.

Thresholds come from an A/A control run -- identical code on both sides.
Run time stayed within 2%, so 5% is the band. Compile time swung 20% on
one benchmark, and 216% on one with no jitted work at all, hence the
looser 25% band and the 50 ms floor below which a cold/warm gap is
warm-up rather than compilation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: format `benchmark.yml`

* fix: post the benchmark report as a PR comment, not just an artifact

The report was only reaching the job summary and the artifact. The
workflow meant to comment it ne... (continued)

14611 of 15819 relevant lines covered (92.36%)

1.69 hits per line

Jobs
ID Job ID Ran Files Coverage
1 test-inference - 30701977069.1 86
71.08
 GitHub Action Run
2 test-modeling - 30701977069.2 86
68.73
 GitHub Action Run
3 examples - 30701977069.3 86
28.81
 GitHub Action Run
Source Files on build 30701977069