apache / carbondata / 917

76%

push

jenkins

[CARBONDATA-2885] Broadcast Issue and Small file distribution Issue

Issue :-

In External Table Carbon Relation sizeInByte is wrong (always 0) because of this Join Queries are identified for broadcast even
Table actual size is > 10MB( default broadcast).This is making fail some of the join table ( table which should select sortmergeJoin but
because of wrong calculation it gone for broadcast join)

if Merge_small_file task distribution is enabled ,Join queries are failed (TPCH).
carbon opens many carbon files but it not getting closed.

Root Cause :-
1. Current relation size calculation is based on tablestatus file but since
External Table does not have tablestatus file so always zero was returned.
2. if Merge_small_file task distribution is enabled carbon opens many carbon files but it not getting closed.
Solution :-

if Table is External Table then calculate size from TablePath .
close the carbon files for scan is finished.

This closes #2658

30 of 30 new or added lines in 2 files covered. (100.0%)

57372 of 71115 relevant lines covered (80.67%)

1.07 hits per line