apache / parquet-cpp / 561

push

travis-ci

PARQUET-436: Implement basic Write Support

Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up.

One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here.

Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them.

Author: Uwe L. Korn <uwelk@xhochy.com>

Closes #90 from xhochy/parquet-436 and squashes the following commits:

ebfc3e0 [Uwe L. Korn] Address review comments
4a81afe [Uwe L. Korn] Address review comments
80af8ca [Uwe L. Korn] Address review comments
0cf2aa1 [Uwe L. Korn] Address code review comments
77da3c7 [Uwe L. Korn] Style fixes
afbb6da [Uwe L. Korn] Lint fixes
04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs
02e0ba5 [Uwe L. Korn] More code cleanup
36b7d1d [Uwe L. Korn] Code cleanups
b581da1 [Uwe L. Korn] Get rid of num_nulls
f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels
e306b04 [Uwe L. Korn] Running write unit test
ad351e8 [Uwe L. Korn] Added more tests for PageWriter
63c06f0 [Uwe L. Korn] Refactored unit tests
ebd9648 [Uwe L. Korn] We can now at least open a self-written file
7bedf34 [Uwe L. Korn] Lint fixes
aa84251 [Uwe L. Korn] Add support for writing repetition levels
b354e07 [Uwe L. Korn] Add support for writing definition levels
7298f96 [Uwe L. Korn] Lint fixes
9fe1d89 [Uwe L. Korn] Some unit test refactorings
9e21c56 [Uwe L. Korn] First version that can write a page
0662386 [Uwe L. Korn] First draft of write API

418 of 418 new or added lines in 10 files covered. (100.0%)

4557 of 4787 relevant lines covered (95.2%)

22258.93 hits per line