ConnectionMaster / json / 31042126870

97%

push

github

Fix start_pos() for strings containing escape sequences (#5361)

The diagnostic position of a string value was derived by subtracting the
parsed value's length from the end position. Escape sequences make the
source token longer than the value it parses to, so the reported start
position landed inside the string, one byte off per escape sequence:

    input: {"a":"\n\n\n\n\n\n"}
      start_pos() == 11, so the reported range covered  n\n\n\n"
      instead of the documented "\n\n\n\n\n\n"

This contradicts the documented behavior of start_pos(), which is the
position of the opening quote, and it also corrupted the "(bytes N-M)"
part of JSON_DIAGNOSTICS exception messages. Strings with multi-byte
UTF-8 but no escapes were unaffected, which is why this went unnoticed.

Record the offset of the token in the lexer when it starts scanning and
use that, instead of reconstructing it from the parsed value. Booleans,
null and numbers already reported correct positions and are unchanged.

The new lexer member and accessor are compiled only when
JSON_DIAGNOSTIC_POSITIONS is enabled, which is already part of the ABI
tag, so the default build is unaffected.

Signed-off-by: Niels Lohmann <mail@nlohmann.me>

2176 of 2340 branches covered (92.99%)

Branch coverage included in aggregate %.

5 of 5 new or added lines in 2 files covered. (100.0%)

6679 of 6765 relevant lines covered (98.73%)

16266157.94 hits per line