amazon-aurora / babelfish_extensions / 15711277723

75%

main: 77%

push

github

Do not coerce float/real datatypes in pre_transform_hook (#3840)

ISSUE: When lower precedence datatypes (to real/float) are grouped on one side and then union'ed with real or float datatypes then "not equal" values were being grouped together. This was found during real and float testing when "not equal" real values were being grouped together and the root cause helped us extrapolate issues with numeric/int/smallint/tinyint as well.
NOTE: In Fact, for Numeric unioned REAL/FLOAT we were also observing crashes due to the early coercion and precision mismatch during aggregation.

Root Cause: The coerce_to_target_type during pre_transform_hook is applying the implicit cast too early -- even before grouping.
When the REAL(or other lower precedence type) values are cast to FLOAT(Or Higher precedence types) during the coercion, there might be precision differences. The cast types in PG from lower to high precedence types are defined as "IMPLICIT" which gives the freedom to optimizer to evaluate on the go. In case of "ASSIGNED" cast types optimizer would have grouped first and then applied cast -- it prevents the automatic type conversion during the grouping phase -- which would be the expected outcome.

Solution: Prevent the cast from happening too early for REAL/FLOAT, thus, maintaining the original precision for grouping and only cast for the final result.

Issues Resolved:
BABEL-5798

Signed-off-by: kushaal shroff <kushaal@amazon.com>

3 of 3 new or added lines in 1 file covered. (100.0%)

48641 of 64481 relevant lines covered (75.43%)

320447.06 hits per line