tarantool / tarantool / 12480240239

87%

master: 87%

push

github

cmake: shuffle testcases in luatest's tests

luatest has an option `--shuffle` that allows to reorder testcases
before execution. This can be useful to detect a test that passes
just because it happens to run after an unrelated test that leaves
the system in a favourable state. Patch enables tests shuffling
with the predefined random seed generated by CMake [1]. Seed is
useful for reproducing a problems related to the specific test
order.

The patch enables shuffle mode only in `app-luatest`,
`metrics-luatest`, `config-luatest` and `vinyl-luatest`. In other
test suites tests failed with enabled shuffle mode. For these
suites mode will be enabled later.

Follows up commit 9e3e9eedc025
("cmake: support luatest tests by CTest")

1. https://cmake.org/cmake/help/latest/command/string.html#random

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

(cherry picked from commit 8af038b00)

69685 of 123524 branches covered (56.41%)

102630 of 117464 relevant lines covered (87.37%)

3179407.33 hits per line