exasol / cloud-storage-etl-udfs

97%

master: 95%

LAST BUILD ON BRANCH feature/import-parquet-int64-millis

branch: feature/import-parquet-int64-millis

CHANGE BRANCH

x

Reset

• feature/import-parquet-int64-millis
• bug/#56-orc-import-should-be-java-bigdecimal
• bug/#65-fix-regex-path-matching-error
• develop
• documentation/#97-update-docs
• enhancement/72_system_requirements_and_design
• enhancement/add_logos
• feature/#31-delta-format-support
• feature/#36-overwrite-bucket-path-if-required
• feature/#58-azure-datalake-gen2
• feature/74_create_initial_implementation_of_kinesis_connector
• feature/87_kinesis_nested_data_support
• feature/add-connection-object
• fix/links
• improvement/#50-obtain-account-container-names-from-path
• improvement/#51-refactor-style-lint-checks
• improvement/#82-compression-codec-extension
• master
• refactoring/#98-remove-kafka-kinesis-modules
• refactoring/70_update_dependencies_for_ide
• task/#45-update-docs
• task/#63-add-dependency-list
• task/#84-change-to-multi-project-setup
• task/benchmarks
• v0.5.0
• v0.6.0
• v0.6.0-7-g451d30c-SNAPSHOT
• v0.6.1
• v0.6.2
• v0.6.2-rc1
• v0.7.0
• v0.7.1
• v0.8.0
• v0.8.0-rc1

Build # 119

Build Type

Pull #53

travis-ci

Committed by web-flow

Commit Message

Correctly import Parquet timestamps encoded as milliseconds since epoch.

So far, we could import Parquet timestamp values that are encoded as
`INT96` binary type. This type encodes timestamps as 12-byte binary with
nanosecond precision.

However, it is possible to also encode timestamp as milliseconds since
epoch using `INT64` type.

For example,

```
message test_timestamps {
  required int64 col_long;
  required int64 col_timestamp (TIMESTAMP_MILLIS);
}
```

the first column contains Long value and the second column millis since
epoch as Long value, but should be imported as Timestamp value.

Previously the second column value assumed to be INT96 binary which was
a bug.

Fixes #52.

Pull Request Pull Request #53: Fix Parquet import timestamps encoded as milliseconds since epoch.

Run Details

5 of 5 new or added lines in 2 files covered. (100.0%)

745 of 767 relevant lines covered (97.13%)

1.94 hits per line