24562565037
58%
master: 62%

Ran 17 Apr 2026 11:26AM UTC

Jobs 1

Files 40

Run time 1min

Badge

Embed ▾

Committed 17 Apr 2026 11:22AM UTC coverage: 58.057%. Remained the same

Build # 24562565037

Build Type

Pull #3434

github

Committed by

XploY04

Commit Message

fix(e2e): bump Megatron notebook Complete-wait timeout to 10m

The Megatron TP notebook waits for the TrainJob to reach Complete with
timeout=120. On the oracle-vm-gpu-a10-1 CI runner the happy-path time
from TrainJob creation to Complete is ~96s (measured on the last passing
GPU E2E run, 2026-04-15). Any runner slowdown, image-pull delay, or
GPU-advertisement latency on top of that pushes the test past the 120s
budget and papermill raises TimeoutError, even though the TrainJob is
still on track to finish.

Every GPU E2E run has been failing here since 2026-04-16 ~16:00 UTC on
every branch, with no functional repo change between the last pass and
first fail. Bumping to 600s gives enough headroom for cold image pulls
and scheduling variance without masking real failures (papermill's
outer --execution-timeout is 1800s).

Signed-off-by: XploY04 <2004agarwalyash@gmail.com>

Pull Request Pull Request #3434: fix(examples): bump Megatron notebook Complete-wait timeout to 10m

Coverage Stats

2032 of 3500 relevant lines covered (58.06%)

0.67 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
1	24562565037.1	17 Apr 2026 11:26AM UTC	40	58.06	GitHub Action Run

kubeflow / trainer / 24562565037
58%
master: 62%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 24562565037

kubeflow / trainer / 24562565037 58% master: 62%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 24562565037

kubeflow / trainer / 24562565037
58%
master: 62%

README BADGES
x