tarantool / tarantool / 12478853495

88%

push

github

cmake: shuffle testcases in luatest's tests

luatest has an option `--shuffle` that allows to reorder testcases
before execution. This can be useful to detect a test that passes
just because it happens to run after an unrelated test that leaves
the system in a favourable state. Patch enables tests shuffling
with the predefined random seed generated by CMake [1]. Seed is
useful for reproducing a problems related to the specific test
order.

The patch enables shuffle mode only in `app-luatest`,
`metrics-luatest`, `config-luatest` and `vinyl-luatest`. In other
test suites tests failed with enabled shuffle mode. For these
suites mode will be enabled later.

Follows up commit 9e3e9eedc025
("cmake: support luatest tests by CTest")

1. https://cmake.org/cmake/help/latest/command/string.html#random

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

69666 of 123518 branches covered (56.4%)

102636 of 117460 relevant lines covered (87.38%)

3036028.16 hits per line