tarantool / luajit / 6839606957

71%

tarantool/master: 93%

push

github

cmake: replace prove with CTest

The patch replaces the main test runner prove(1) with CTest,
see [1] and [2].

Build and test steps looks following:

$ cmake -S . -B build
$ cmake --build build --parallel
$ cmake --build --target [LuaJIT-test, test]

CMake targets lua-Harness-tests, LuaJIT-tests, PUC-Rio-Lua-5.1-tests,
tarantool-tests and tarantool-c-tests are still functional.

The benefits are:

- less external dependencies, prove(1) is gone
- running tests by test title
- extended test running options in comparison to prove(1)
- unified test output (finally!)

Note that it is not possible to attach targets to the automatically
generated `test` target. It means that target `test` is now executing
`LuaJIT-test`, but not `LuaJIT-lint`.

Note that the testsuites in `test/LuaJIT-tests` and
`test/PUC-Rio-Lua-5.1` directories have their own test runners written
in Lua and currently CTest runs these testsuites as single tests. In the
future we could change these test runners to specify tests from outside,
and after this we could run tests separately by CTest in these test
suites.

Note that it is not possible to add dependencies to `add_test()` in
CMake, see [3]. CMake 3.7 introduces FIXTURES_REQUIRED [4] and
FIXTURES_SETUP [5], but these test properties cannot be used - used
CMake version is lower. This workarounded by introducing a special test
for each testsuite that executes a target that builds test dependencies.

1. https://cmake.org/cmake/help/latest/manual/ctest.1.html
2. https://cmake.org/cmake/help/book/mastering-cmake/chapter/Testing%20With%20CMake%20and%20CTest.html
3. https://gitlab.kitware.com/cmake/cmake/-/issues/8774
4. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_REQUIRED.html
5. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_SETUP.html

5341 of 5972 branches covered (0.0%)

Branch coverage included in aggregate %.

20479 of 23293 relevant lines covered (87.92%)

2750478.5 hits per line