tarantool / luajit / 6605130947

71%

tarantool/master: 93%

push

github

cmake: replace prove with CTest

The patch replaces the main test runner prove(1) with CTest [1].

The benefits are:

- less external dependencies, prove(1) is gone
- running tests by test title
- extended test running options in comparison to prove(1)
- unified test output (finally!)

Note, it is not possible to attach targets to the automatically
generated `test` target. It means that target `test` now is executing
`LuaJIT-test`, but not `LuaJIT-lint`.

Note, the testsuites in `test/LuaJIT-tests` and in
`test/PUC-Rio-Lua-5.1` directories have their own test runners and
currently CTest runs these testsuites as a single tests. In a future we
could change these test runners to specify test from outside and after
this we could run tests separately by CTest in these test suites.

Note, it is not possible to add dependencies to `add_test()` in CMake,
see [2]. It means that all object files and executables must be build
before running ctest(1). Users in SO thread [3] recommends wraps
ctest(1) in CMake by another target and use it for running or use
`FIXTURES_SETUP` or `FIXTURES_REQUIRED` when CMake version >= 3.7.
As a workaround one must call target `all` before running `ctest`.

User-visible changes are:

$ cmake -S . -B build
$ cmake --build build --target test --parallel

Using CTest in a build directory:

$ ctest                             # Running all tests.
$ ctest -L tarantool                # Running tests in tarantool-tests dir.
$ ctest -L tarantool-c              # Running tests in tarantool-c-tests dir.
$ ctest -L lua-Harness              # Running tests in lua-Harness dir.
$ ctest -L luajit                   # Running tests in LuaJIT dir.
$ ctest -L lua                      # Running tests in PUC-Rio-Lua-5.1 dir.
$ ctest --rerun-fail
$ ctest -V
$ ctest --output-on-failure

1. https://cmake.org/cmake/help/latest/manual/ctest.1.html
2. https://gitlab.kitware.com/cmake/cmake/-/issues/8774
3. https://stackoverflow.com/questions/733475/cmake-ctes... (continued)

5336 of 5972 branches covered (0.0%)

Branch coverage included in aggregate %.

20444 of 23293 relevant lines covered (87.77%)

2758932.68 hits per line