babelfish-for-postgresql / babelfish_extensions / 15711252052

77%

Build Type

push

github

Committed by web-flow

Commit Message

Do not coerce float/real datatypes in pre_transform_hook (#3840)

ISSUE: When lower precedence datatypes (to real/float) are grouped on one side and then union'ed with real or float datatypes then "not equal" values were being grouped together. This was found during real and float testing when "not equal" real values were being grouped together and the root cause helped us extrapolate issues with numeric/int/smallint/tinyint as well.
NOTE: In Fact, for Numeric unioned REAL/FLOAT we were also observing crashes due to the early coercion and precision mismatch during aggregation.

Root Cause: The coerce_to_target_type during pre_transform_hook is applying the implicit cast too early -- even before grouping.
When the REAL(or other lower precedence type) values are cast to FLOAT(Or Higher precedence types) during the coercion, there might be precision differences. The cast types in PG from lower to high precedence types are defined as "IMPLICIT" which gives the freedom to optimizer to evaluate on the go. In case of "ASSIGNED" cast types optimizer would have grouped first and then applied cast -- it prevents the automatic type conversion during the grouping phase -- which would be the expected outcome.

Solution: Prevent the cast from happening too early for REAL/FLOAT, thus, maintaining the original precision for grouping and only cast for the final result.

Issues Resolved:
BABEL-5798

Signed-off-by: kushaal shroff <kushaal@amazon.com>

Run Details

3 of 3 new or added lines in 1 file covered. (100.0%)

2 existing lines in 1 file now uncovered.

48877 of 64689 relevant lines covered (75.56%)

317038.07 hits per line