tarantool / luajit / 7712549404

89%

tarantool/master: 93%

push

github

Fix zero stripping in %g number formatting.

Reported by pwnhacker0x18.

(cherry picked from commit 343ce0eda)

In the situation when the precision (`prec`) and amount of digits
(`hilen`) for the decimal representation are the same and `ndhi` == 0,
the `ndlo` part will become 64 (the size of the `nd` stack buffer), and
the overflow occurs.

This patch adds the corresponding mask (0x3f == 63) for the `ndlo`
incrementation result.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#9595

5402 of 5997 branches covered (0.0%)

Branch coverage included in aggregate %.

20699 of 23370 relevant lines covered (88.57%)

2745723.86 hits per line