tarantool / luajit / 17634222908

93%

tarantool/master: 93%

push

github

Correctly close VM state after early OOM during open.

Reported by Assumeru.

(cherry picked from commit 5ca25ee83)

`lua_newstate()` sets `g->str.mask` to `~(MSize)0` before calling
`cpluaopen(). If OOM happens before the `lj_str_init()` call,
`lj_gc_freeall()` calls `gc_sweepstr()` in a loop with incorrect top
limit `g->str.mask`, which leads to the crash.

This patch changes the order of the loop iteration with the correct
bottom limit.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#11691

Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 89dd3a96a)

5712 of 6046 branches covered (94.48%)

Branch coverage included in aggregate %.

2 of 2 new or added lines in 1 file covered. (100.0%)

21796 of 23512 relevant lines covered (92.7%)

3873242.22 hits per line