1959

Ran 18 Apr 2018 08:32AM UTC

Jobs 3

Files 0

Run time –

Badge

Embed ▾

pending completion

Build # 1959

Build Type

push

travis-ci

Committed by

xhochy

Commit Message

PARQUET-1273: Properly write dictionary values when writing in chunks

The error was reported here: https://issues.apache.org/jira/browse/ARROW-1938.

Because dictionary types are not supported in writing yet, the code converts the dictionary column to the actual values first before writing. However, the existing code was accidentally using zero as the offset and the length of the column as the size. This resulted in writing all of the column values for each chunk of the column that was supposed to be written.

The fix is to pass the offset and size when recursively calling through to WriteColumnChunk with the "flattened" data.

Author: Joshua Storck <joshua.storck@twosigma.com>

Closes #453 from joshuastorck/ARROW_1938 and squashes the following commits:

c2af50f [Joshua Storck] Remove extraneous semicolon in unit test
23f5722 [Joshua Storck] Ran clang-format on arrow-reader-writer-test.cc
314b159 [Joshua Storck] Removing print statements from AssertTableEqual
f0bc71a [Joshua Storck] Fixing bug reported in https://issues.apache.org/jira/browse/ARROW-1938, namely preventing all of the values in a dictionary column from being written to parquet for each chunk created as a result of specifying row_group_size

Run Details

Jobs

ID	Job ID	Ran
1	1959.1	18 Apr 2018 08:33AM UTC	Travis Job 1959.1
2	1959.2	18 Apr 2018 08:32AM UTC	Travis Job 1959.2
3	1959.3	18 Apr 2018 08:32AM UTC	Travis Job 1959.3

Source Files on build 1959

Detailed source file information is not available for this build.

apache / parquet-cpp / 1959

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 1959

apache / parquet-cpp / 1959

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 1959

README BADGES
x