1
76%
master: 76%

Ran 09 Oct 2018 10:06AM UTC

Files 1014

Run time 41s

Badge

Embed ▾

Committed 09 Oct 2018 10:06AM UTC coverage: 80.615% (+0.02%) from 80.592%

Job # 1153.1

Build Type

push

jenkins

Committed by ravipesala

Commit Message

[CARBONDATA-2990] Queries slow down after some time due to broadcast issue

Problem
It is observed that during consecutive run of queries after some time queries are slowing down. This is causing the degrade in query performance.
No exception is thrown in driver and executor logs but as observed from the logs the time to broadcast hadoop conf is increasing after every query run.

Analysis

This is happening because in carbon SerializableConfiguration class is overriden from spark. Spark registers this class with Kryo serializer and hence the computation using the kryo is fast. The same benefit is not observed in carbondata becuase of overriding the class.
Internal Spark sizeEstimator calculates the size of object and there are few extra objects in carbondata overriden class because of which the computation time is increasing.
Solution
Use the spark class instead of overriding the class in carbondata

This closes #2803

Coverage Stats

60631 of 75211 relevant lines covered (80.61%)

1.06 hits per line

apache / carbondata / 1153 / 1
76%
master: 76%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 1153.1

apache / carbondata / 1153 / 1 76% master: 76%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 1153.1

apache / carbondata / 1153 / 1
76%
master: 76%

README BADGES
x