Warning: This build has drifted.
The coverage report for this pull request build may be inaccurate because its base commit is no longer the HEAD of its target branch.
This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

    • Learn more: For more information on this, see Tracking coverage changes for pull request builds.
    • Fix now: For a quick fix, rebase this PR at GitHub. Your next report should be accurate.
    • Prevent going forward: To avoid this issue with future PRs, see these Recommended CI Configurations.

fix(runtimes): wait for launcher DNS before mpirun in MPI runtimes

The MLX and DeepSpeed ClusterTrainingRuntimes are MPI-based. #3748 made
the launcher depend on the node replicated job reaching Ready, so the
workers pass their sshd readiness probe and get their DNS records
published before the launcher starts. The launcher then runs mpirun
immediately, and orted on each worker connects back to the launcher
FQDN (<jobset>-launcher-0-0.<jobset>). That record is created later than
the workers' and can lag behind mpirun startup, so orted cannot route
back and the launcher exits with "ORTE was unable to reliably start one
or more daemons". launcher-to-worker resolution already works; only the
worker-to-launcher direction races.

Add a wait-for-launcher-dns initContainer to the launcher pod that
blocks until the launcher's own FQDN resolves via getent hosts before
the main container runs mpirun. This guarantees bidirectional DNS
resolution without the -mca routed direct workaround, which would paper
over the missing record rather than wait for it. The subdomain equals
the JobSet name, injected via the downward API from the
jobset.sigs.k8s.io/jobset-name label.

Applied to both the kustomize base manifests (used by the GPU E2E job)
and the Helm chart runtime files to keep them in sync.

Signed-off-by: Sanskar Singh <sanskarsinghty1234@gmail.com>