tarantool / luajit / 7799094684

70%

tarantool/master: 93%

push

github

Fix zero stripping in %g number formatting.

Reported by pwnhacker0x18.

(cherry picked from commit 343ce0eda)

LuaJIT uses `lj_strfmt_wfnum()` for number formatting. In the situation
when the precision (`prec`) and amount of digits (`hilen`) for the
decimal representation are the same and `ndhi` == 0, the `ndlo` part
becomes 64 (the size of the `nd` stack buffer), and the overflow occurs.

This patch adds the corresponding mask (0x3f == 63) for the `ndlo`
incrementation result.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9595

Reviewed-by: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>
(cherry picked from commit 15acf5b2f)

5387 of 5981 branches covered (0.0%)

Branch coverage included in aggregate %.

20622 of 23282 relevant lines covered (88.57%)

696153.5 hits per line