IBM / unitxt / 42386153629

78%

push

github

CI: Make the internal LLMEvalKit install diagnosable, and skip it when GHES is unreachable (#1999)

* CI: Make the internal LLMEvalKit install step diagnosable

A failure in `.github/actions/install-internal-pip` was almost unreadable.
GitHub masks the step's output because it carries a secret, and the step ran
under `-e -o pipefail`, so any failing command aborted it with no output at all:

    ##[error]Process completed with exit code 1

Five distinct causes -- absent secret, malformed secret, unreachable GHES,
rejected credential, broken package -- all looked exactly alike. This is not
hypothetical: it is how CI is failing right now. `unittests`, `eager`,
`performance` and all ten `preparation` shards die in this step, on main as
well as on every open PR, and the only way to tell that the cause is network
reachability rather than the code under test is to notice that the
unconditional `echo "Installing from URL: ..."` never printed, so execution
must have stopped in the three commands before it.

Each precondition is now checked separately and fails with its own `::error`
annotation, which shows in the job summary without opening the log:

  - Malformed SSH key   -- `ssh-keygen` cannot load the secret
  - GHES unreachable    -- `ssh-keyscan -T 15` got no host key
  - SSH key rejected    -- the host refused the deploy key
  - Internal install failed -- host and credential fine, pip failed
  - pydantic pin failed -- the package installed, the pin did not

Each message says which external dependency broke and where to look, and the
two infrastructure cases say outright that the pull request is not at fault.

Implementation notes:

`set -e` is dropped deliberately. Every stage now reports before it exits, so
the step can no longer abort without an explanation; `-o pipefail` is kept.

The auth probe matches ssh's *message*, not its exit status: a successful
`ssh -T git@github.com` exits 1 and prints "Hi <user>! You've successfully
authenticated". Keying... (continued)

2772 of 4026 branches covered (68.85%)

Branch coverage included in aggregate %.

10906 of 13569 relevant lines covered (80.37%)

0.8 hits per line