tarantool / luajit / 22904062882 / 1

70%

tarantool/master: 93%

push

github

Fix edge cases when recording string.byte/sub.

Thanks to Sergey Kaplun.

(cherry picked from commit 89f268b3f)

It is possible that the `len` (`end - start`) will underflow and become
positive in the `recff_string_range()` when the `end` is negative. For
`string.sub()` this is not crucial, since the trace will be valid
anyway. But for `string.byte()` it may lead to the assertion failure in
the `rec_check_slots()`.

This patch fixes those underflows by the correct comparison.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#12134

Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Signed-off-by: Sergey Kaplun <skaplun@tarantool.org>
(cherry picked from commit b45b65c33)

5690 of 6027 branches covered (94.41%)

Branch coverage included in aggregate %.

21701 of 23424 relevant lines covered (92.64%)

1854457.03 hits per line