skeema / skeema / 14583325066

92%

push

github

diff/push: improve char vs text length comparison logic

When converting a column type between char/varchar and a text type, an
existing safety check compares the maximum string length that is storable
in the old and new column definitions, and flags the change as unsafe if the
new length is potentially insufficient to store all old values. However, the
logic failed to account for multibyte characters properly: the maximum storage
of text types is listed in the MySQL/MariaDB manuals in terms of *bytes*, not
characters. For example, tinytext can store 255 bytes, but with utf8mb4
this is insufficient to safely store a varchar(64) consisting of all 4-byte
chars, so an ALTER TABLE attempting such a conversion is now flagged as
unsafe.

The real-life effect of this logic change is relatively minor, which is why
the discrepancy was never caught previously. It largely just affects
conversions to/from tinytext, as the other text types are sufficiently large
to always store the maximum-allowed char/varchar regardless of charset. And
with strict sql_mode, overly-large values will be flagged by the server and
the ALTER TABLE won't proceed anyway.

This commit also introduces helper logic that makes it easier to work with
charsets in general, and tracks flavor-specific differences in them.

40 of 50 new or added lines in 3 files covered. (80.0%)

10395 of 11160 relevant lines covered (93.15%)

1.11 hits per line