1
76%
master: 76%

Ran 17 Jun 2019 05:43PM UTC

Files 1061

Run time 1min

Badge

Embed ▾

Committed 17 Jun 2019 05:43PM UTC coverage: 77.318% (+0.002%) from 77.316%

Job # 1772.1

Build Type

push

jenkins

Committed by xubo245

Commit Message

[CARBONDATA-3415] Merge index is not working for partition table. Merge index for partition table is taking significantly longer time than normal table.

Problem:
(1) Merge index is not working for partition table.
(2) Merge index for partition table is significantly more than the normal carbon table.

Root cause:
(1) Merge index event listener is moved to preStatusUpdateEvent in #3221 . But preStatusUpdateEvent is not triggered in case of partition table. Test case to validate merge index on partition table is also wrong. Not caught in the test builders.
(2) Currently, merge index job will trigger tasks like one segment one task. But for a partition table, there are partitions in a segments and merge index is for partitions. So per segment it has to iterate and merge the index files inside partitions, because of this the time is little more when the number of partitions are high. Number of Tasks = Number of Segments.

Solution:
(1) Correct the test case and trigger merge index listener for partition table.
(2) Parallelize the tasks launched to the partitions. Number of tasks = Number of partitions in a segment

This closes #3262

Coverage Stats

64081 of 82880 relevant lines covered (77.32%)

1.01 hits per line

apache / carbondata / 1772 / 1
76%
master: 76%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 1772.1

apache / carbondata / 1772 / 1 76% master: 76%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 1772.1

apache / carbondata / 1772 / 1
76%
master: 76%

README BADGES
x