tarantool / luajit / 22765432116 / 1

93%

tarantool/master: 93%

push

github

Fix edge cases when recording string.byte/sub.

Thanks to Sergey Kaplun.

(cherry picked from commit 89f268b3f)

It is possible that the `len` (`end - start`) will underflow and become
positive in the `recff_string_range()` when the `end` is negative. For
`string.sub()` this is not crucial, since the trace will be valid
anyway. But for `string.byte()` it may lead to the assertion failure in
the `rec_check_slots()`.

This patch fixes those underflows by the correct comparison.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#12134

5711 of 6046 branches covered (94.46%)

Branch coverage included in aggregate %.

21786 of 23513 relevant lines covered (92.66%)

3868112.54 hits per line