vispy / vispy / 9484800458

75%

push

github

Fix CI: failing tests and linting errors not failing checks (#2597)

I think I've sorted out the problem causing #2596. Sorry this is a lot
to take in. I'm just trying to be thorough and keep track for my own
purposes as well.

Before I write any more I'll try to be clear:
* _tests_ are the individual test functions in the test suite
* _checks_ are the the output of GitHub actions workflows
* _steps_ are the individual steps in a workflow
* _commands_ are the individual lines in a `run` step

### Tests
The issue was that the GitHub actions workflow is replacing the default
shell with `bash -l {0}`.

https://github.com/vispy/vispy/blob/<a class=hub.com/vispy/vispy/commit/4f061ba23ddac3131eb0d1d955573dd71c9a3d2f">4f061ba23/.github/workflows/main.yml#L163-L166

This is problematic because it does not fail a run step if a command
fails. I've changed the shell to `bash --login -e {0}` which will fail
the run step if any command fails. The default shell on GitHub actions
sets `-e`, and I think it's what we want. Without this, the step (and
check) will only fail if the _last_ command in the step fails. I didn't
test, but I believe the login shell may be required for some conda
things.

Many checks would fail not directly because the test failed, but because
the subsequent coverage commands would fail (because the tests failed in
a way that didn't produce coverage).

https://github.com/vispy/vispy/blob/4f061ba23ddac3131eb0d1d955573dd71c9a3d2f/.github/workflows/main.yml#L274-L275

Because of all this, I think the only tests that would actually be
missed were any marked
`@requires_pyopengl()` but not `@requires_application()` (as the in the
case in #2589).

Documenting the process in this PR:
* See this log showing a check that passed, but should have failed
(5964e1e12):

https://github.com/aganders3/vispy/actions/runs/9388966425/job/25855466353#step:11:647
* See here for a check that failed as expected, after changing the shell
(8a314d... (continued)

5792 of 10518 branches covered (55.07%)

23958 of 32047 relevant lines covered (74.76%)

2.45 hits per line