apache / carbondata / 1198
76%

DEFAULT BRANCH: master
Ran
Jobs 1
Files 1015
Run time 50s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
1198

push

jenkins

xuchuanyin 
[CARBONDATA-3040][BloomDataMap] Fix bug for merging bloom index

Problem
There is a bug which causes query failure when we create two bloom datamaps on same table with data.

Analyze
Since we already have data, each create datamap will trigger rebuild datamap task and then trigger bloom index file merging. By debuging, we found the first datamap's bloom index files would be merged two times and the second time made bloom index file empty.

The procedure goes as below:

  1. create table
  2. load data
  3. create bloom datamap1: rebuild datamap1 for existing data, event listener is trigger to merge index files for all bloom datamaps( currently only datamap1 )
  4. create bloom datamap2: rebuild datamap2 for existing data, event listener is trigger to merge index files for all bloom datamaps (currently datamap1 and datamap2)

Because the event does not has information which datamap it rebuilt, it always rebuilds all bloom datamap. So datamap1's bloom index files would be merged 2 times, but only remains a mergeShard folder when it ran the second merged such that no file input for merging and the final merge bloom index files are empty.

Solution
Send the datamap name in rebuild event for filter and only merge bloom index files for the specific datamap. Also add file check whether mergeShard already exists before merging.

This closes #2851

60522 of 75042 relevant lines covered (80.65%)

1.06 hits per line

Jobs
ID Job ID Ran Files Coverage
1 1198.1 0
80.65
Source Files on build 1198
Detailed source file information is not available for this build.