tarantool / luajit / 15492086894 / 1

93%

tarantool/master: 93%

push

github

Rework stack overflow handling.

Reported by pwnhacker0x18. Fixed by Peter Cawley.

(cherry picked from commit defe61a56)

In case of the Lua stack overflow error, LuaJIT restores the `L->top`
value and pushes the error message above. It is possible that the
restored value is greater than `L->maxstack`, so pushing the error
message causes dirty write out-of-bounds.

This patch prevents it by overwriting stack overflow handling machinery.
Now, in the aforementioned case, the last frame is replaced with a dummy
frame to avoid dirty writes. In some cases, there may not be enough
space on the stack to invoke the error handler. See the related changes
in the <test/LuaJIT-tests/lang/stackov.lua>.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#11278

5710 of 6045 branches covered (94.46%)

Branch coverage included in aggregate %.

21793 of 23506 relevant lines covered (92.71%)

3834446.44 hits per line