apache / parquet-cpp / 733 / 1

Ran 03 Aug 2016 12:37AM UTC

Files 108

Run time 5s

Badge

Embed ▾

Committed 02 Aug 2016 11:38PM UTC coverage: 97.133% (+1.6%) from 95.512%

Job # 733.1

Build Type

push

travis-ci

Committed by Wes McKinney

Commit Message

PARQUET-671: performance improvements for rle/bit-packed decoding

Testing on my own data shows an order-of-magnitude improvement.

I separated the commits for clarity, each one gives an imcremental improvement.

The motivation for the last commit (allowing NULL for def_levels/rep_level) is a workaround for Spark  which doesn't seem to be able to generate columns without def_level, even when a column is specified as "not nullable".

Author: Eric Daniel <edaniel@comscore.com>

Closes #140 from edani/decode-perf and squashes the following commits:

eec0855 [Eric Daniel] Ran "make format"
0568de6 [Eric Daniel] Only check num. of repetition levels when def_levels is set
5f54e1c [Eric Daniel] Added benchmarks for dictionary decoding
087945b [Eric Daniel] Style fixes from code review
906be73 [Eric Daniel] Allow the reader to skip rep/def decoding
04b7391 [Eric Daniel] Fast bit unpacking
bda5d84 [Eric Daniel] The bit reader can decode in batches
3f10378 [Eric Daniel] Improve decoding of repeated values in the dict encoding

Run Details

7962 of 8197 relevant lines covered (97.13%)

11712.74 hits per line

apache / parquet-cpp / 733 / 1

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 733.1

apache / parquet-cpp / 733 / 1

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 733.1

README BADGES
x