tarantool / luajit / 7475489974

71%

tarantool/master: 93%

push

github

cmake: replace prove with CTest

The patch replaces the main test runner `prove(1)` with CTest,
see [1] and [2].

Build and test steps looks the same as before:

$ cmake -S . -B build
$ cmake --build build --parallel
$ cmake --build build --target [LuaJIT-test, test]

CMake targets lua-Harness-tests, LuaJIT-tests, PUC-Rio-Lua-5.1-tests,
tarantool-tests and tarantool-c-tests are still functional.

One could use `ctest` tool:

$ ctest --print-labels
Test project <snipped>
All Labels:
  LuaJIT
  PUC-Rio-Lua-5.1
  lua-Harness
  tarantool
  tarantool-c
$ ctest                             # Run all tests.
$ ctest -L tarantool                # Run tests in tarantool-tests dir.
$ ctest -L tarantool-c              # Run tests in tarantool-c-tests dir.
$ ctest -L lua-Harness              # Run tests in lua-Harness-tests dir.
$ ctest -L luajit                   # Run tests in LuaJIT-tests dir.
$ ctest -L PUC-Rio-Lua-5.1          # Run tests in PUC-Rio-Lua-5.1-tests dir.
$ ctest --rerun-fail
$ ctest --output-on-failure

The benefits are:

- less external dependencies, `prove(1)` is gone, `ctest` is a part of CMake
- running tests by test title
- extended test running options in comparison to prove(1)
- unified test output (finally!)

Note that it is not possible to attach targets to the automatically
generated `test` target. It means that target `test` is now executing
`LuaJIT-test`, but not `LuaJIT-lint`.

Note that the testsuites in `test/LuaJIT-tests` and
`test/PUC-Rio-Lua-5.1` directories have their own test runners written
in Lua and currently CTest runs these testsuites as single tests. In the
future we could change these test runners to specify tests from outside,
and after this we could run tests separately by CTest in these test
suites.

Note that it is not possible to add dependencies to `add_test()` in
CMake, see [3]. CMake 3.7 introduces FIXTURES_REQUIRED [4] and
FIXTURES_SETUP [5], but these test properties cannot be used - used
CMake version is lowe... (continued)

5374 of 5986 branches covered (0.0%)

Branch coverage included in aggregate %.

20607 of 23353 relevant lines covered (88.24%)

2750123.35 hits per line