1
76%
master: 76%

Ran 08 Jan 2019 09:22AM UTC

Files 1024

Run time 51s

Badge

Embed ▾

Committed 08 Jan 2019 09:22AM UTC coverage: 79.115% (-0.03%) from 79.142%

Job # 1440.1

Build Type

push

jenkins

Committed by ravipesala

Commit Message

[CARBONDATA-3219] Support range partition the input data for local_sort/global sort data loading

For global_sort/local_sort table, load data command add RANGE_COLUMN option

load data inpath '<path>' into table <table name>
options('RANGE_COLUMN'='<a column>')
when we know the total size of input data, we can calculate the number of the partitions.
load data inpath '<path>' into table <table name>
options('RANGE_COLUMN'='<a column>', 'global_sort_partitions'='10')
when we don't know the total size of the input data, we can give the size of each partition.
load data inpath '<path>' into table <table name>
options('RANGE_COLUMN'='<a column>', 'scale_factor'='10')
it will calcute the number of the partitions as follows.

splitSize =  Math.max(blocklet_size, (block_size - blocklet_size)) * scale_factor
numPartitions = Math.ceil(total size / splitSize)
Limitation:

not support insert into, support only load data command,
not support multiple range columns, support only one range column
exists data skew

This closes #2971

Coverage Stats

62214 of 78637 relevant lines covered (79.12%)

1.05 hits per line

apache / carbondata / 1440 / 1
76%
master: 76%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 1440.1

apache / carbondata / 1440 / 1 76% master: 76%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 1440.1

apache / carbondata / 1440 / 1
76%
master: 76%

README BADGES
x