tarantool / luajit / 15491957606 / 1

93%

tarantool/master: 93%

push

github

Fix JIT slot overflow during up-recursion.

Reported by Sergey Kaplun.

(cherry picked from commit 048972dbf)

In the case when LuaJIT is recording the side trace after the
up-recursion call, there is no check that the updated `maxslot` value
doesn't overflow the `LJ_MAX_JSLOTS` limit. If it records several huge
returns in a row, the overflow of the aforementioned limit may occur.
This triggers an assertion failure in `rec_check_slots()`.

This patch fixes it by adding the corresponding check in the
`lj_record_ret()`.

Sergey Kaplun:
* added the description and the test for the problem

Resolves tarantool/security#145
Part of tarantool/tarantool#11278

Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>

5703 of 6042 branches covered (94.39%)

Branch coverage included in aggregate %.

21750 of 23491 relevant lines covered (92.59%)

3831943.23 hits per line