tarantool / luajit / 22904062713 / 1

93%

tarantool/master: 93%

push

github

Fix edge cases when recording string.byte/sub.

Thanks to Sergey Kaplun.

(cherry picked from commit 89f268b3f)

It is possible that the `len` (`end - start`) will underflow and become
positive in the `recff_string_range()` when the `end` is negative. For
`string.sub()` this is not crucial, since the trace will be valid
anyway. But for `string.byte()` it may lead to the assertion failure in
the `rec_check_slots()`.

This patch fixes those underflows by the correct comparison.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#12134

Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit b45b65c33)

5711 of 6046 branches covered (94.46%)

Branch coverage included in aggregate %.

21802 of 23516 relevant lines covered (92.71%)

3890467.23 hits per line