1
76%
master: 76%

Ran 21 May 2018 03:38PM UTC

Files 1011

Run time 46s

Badge

Embed ▾

Committed 21 May 2018 03:38PM UTC coverage: 75.473% (+3.3%) from 72.195%

Job # 465.1

Build Type

push

jenkins

Committed by ravipesala

Commit Message

[CARBONDATA-2433] [Lucene GC Issue] Executor OOM because of GC when blocklet pruning is done using Lucene datamap

Problem
Executor OOM because of GC when blocklet pruning is done using Lucene datamap

Analysis
While seraching using lucene it creates a PriorityQueue to hold the documents. As size is not specified by default the PriorityQueue size is
equal to the number of lucene documents. As the docuemnts start getting added to the heap the GC time increases and after some time task fails due
to excessive GC and executor OOM occurs.
Reference blog: http://lucene.472066.n3.nabble.com/Optimization-of-memory-usage-in-PriorityQueue-td590355.html

Fix
Specify the limit for first search and after that use the searchAfter API to search in incremental order with gieven PriorityQueue size.

This closes #2267

Coverage Stats

52132 of 69074 relevant lines covered (75.47%)

1.01 hits per line

apache / carbondata / 465 / 1
76%
master: 76%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 465.1

apache / carbondata / 465 / 1 76% master: 76%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 465.1

apache / carbondata / 465 / 1
76%
master: 76%

README BADGES
x