tarantool / luajit / 15508613229 / 1

93%

tarantool/master: 93%

push

github

Rework stack overflow handling.

Reported by pwnhacker0x18. Fixed by Peter Cawley.

(cherry picked from commit defe61a56)

In case of the Lua stack overflow error, LuaJIT restores the `L->top`
value and pushes the error message above. It is possible that the
restored value is greater than `L->maxstack`, so pushing the error
message causes dirty write out-of-bounds.

This patch prevents it by overwriting stack overflow handling machinery.
Now, in the aforementioned case, the last frame is replaced with a dummy
frame to avoid dirty writes. In some cases, there may not be enough
space on the stack to invoke the error handler. See the related changes
in the <test/LuaJIT-tests/lang/stackov.lua>.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#11278

Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit 58dcabe69)

5709 of 6045 branches covered (94.44%)

Branch coverage included in aggregate %.

21788 of 23506 relevant lines covered (92.69%)

3834625.75 hits per line