apache / carbondata / 695

76%

Build Type

push

jenkins

Committed by Venkata Ramana G

Commit Message

[CARBONDATA-2760] Reduce Memory footprint and store size for local dictionary encoded columns

Problem:
Local dictionary encoded page is using unsafevarlenghtcolumn column page which internally maintains offset of each value in another column page because of this memory footprint is high.
for complex primitive string data type column page while compressing, it is converting to LV even if it is encoded with dictionary values, because of this store size is high.

Solution:
Use UnsafeFixedLength Column page for local dictionary encoded columns
No need to convert to LV during query if local dictionary is present so use UnsafeFixLength Column page

This closes #2529

Run Details

70 of 70 new or added lines in 11 files covered. (100.0%)

56918 of 70071 relevant lines covered (81.23%)

1.08 hits per line