tarantool / luajit / 6835509493

71%

tarantool/master: 93%

push

github

cmake: replace prove with CTest

The patch replaces the main test runner prove(1) with CTest,
see [1] and [2].

Build and test steps looks following:

$ cmake -S . -B build
$ cmake --build build --parallel
$ cmake --build --target [LuaJIT-test, test]

CMake targets lua-Harness-tests, LuaJIT-tests, PUC-Rio-Lua-5.1-tests,
tarantool-tests and tarantool-c-tests are still functional.

The benefits are:

- less external dependencies, prove(1) is gone
- running tests by test title
- extended test running options in comparison to prove(1)
- unified test output (finally!)

Note that it is not possible to attach targets to the automatically
generated `test` target. It means that target `test` is now executing
`LuaJIT-test`, but not `LuaJIT-lint`.

Note that the testsuites in `test/LuaJIT-tests` and
`test/PUC-Rio-Lua-5.1` directories have their own test runners written
in Lua and currently CTest runs these testsuites as single tests. In the
future we could change these test runners to specify tests from outside,
and after this we could run tests separately by CTest in these test
suites.

Note that it is not possible to add dependencies to `add_test()` in
CMake, see [3]. It means that all object files and executables must be
built before running `ctest(1)`. CMake 3.17 introduces FIXTURES_REQUIRED
[4] and FIXTURES_SETUP [5], but these test properties cannot be used -
used CMake version is lower. This workarounded by introducing a special
test for each testsuite that executes a target that builds test
dependencies.

1. https://cmake.org/cmake/help/latest/manual/ctest.1.html
2. https://cmake.org/cmake/help/book/mastering-cmake/chapter/Testing%20With%20CMake%20and%20CTest.html
3. https://gitlab.kitware.com/cmake/cmake/-/issues/8774
4. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_REQUIRED.html
5. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_SETUP.html

5339 of 5972 branches covered (0.0%)

Branch coverage included in aggregate %.

20485 of 23293 relevant lines covered (87.94%)

2750440.48 hits per line