1691
76%

Ran 19 May 2019 05:59PM UTC

Jobs 1

Files 1047

Run time 50s

Badge

Embed ▾

pending completion

Build # 1691

Build Type

push

jenkins

Committed by ravipesala

Commit Message

[CARBONDATA-3367][CARBONDATA-3368] Fix multiple issues in SDK reader

Problem:
[CARBONDATA-3367] OOM when huge number of carbondata files are read from SDK reader

Cause:
Currently, for each carbondata file, one CarbonRecordReader will be created. And list of CarbonRecordReader will be maintained in carbonReader. so even when CarbonRecordReader is closed, the GC will not happen for that reader as list is still referring that object.
so, each CarbonRecordReader needs separate memory , instead of reusing the previous memory.

Solution : Once CarbonRecordReader.close is done, remove it from the list

problem:
[CARBONDATA-3368]InferSchema from datafile instead of index file

cause:
problem : In SDK, when multiple readers were created with same folder location with different file list, for inferschema all the readers refers same index file, which was causing bottle neck and JVM crash in case of JNI call.

solution: Inferschema from the data file mentioned while building the reader.

problem :
Support list interface for projection, when SDK is called from other languages, JNI interface supports only list from other languages. so need to add list interface for projections.

This closes #3197

Run Details

63793 of 80846 relevant lines covered (78.91%)

1.04 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
1	1691.1	19 May 2019 05:59PM UTC	0	78.91

Source Files on build 1691

Detailed source file information is not available for this build.

apache / carbondata / 1691
76%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 1691

apache / carbondata / 1691 76%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 1691

apache / carbondata / 1691
76%

README BADGES
x