apache / carbondata / 605

76%

push

jenkins

[CARBONDATA-2633][BloomDataMap] Fix bugs in bloomfilter for dictionary/sort/date/TimeStamp column

for dictionary column, carbon convert literal value to dict value, then
convert dict value to mdk value, at last it stores the mdk value as
internal value in carbonfile.

for other columns, carbon convert literal value to internal value using
field-converter.

Since bloomfilter datamap stores the internal value, during query we
should convert the literal value in filter to internal value in order to
match the value stored in bloomfilter datamap.

Changes are made:

1.FieldConverters were refactored to extract common value convert methods.
2.BloomQueryModel was optimized to support converting literal value to
internal value.
3.fix bugs for int/float/date/timestamp as bloom index column
4.fix bugs in dictionary/sort column as bloom index column
5.add tests
6.block (deferred) rebuild for bloom datamap (contains bugs that does
not fix in this commit)

This closes #2403

211 of 211 new or added lines in 18 files covered. (100.0%)

47872 of 72100 relevant lines covered (66.4%)

0.92 hits per line