3068

Ran 17 Oct 2017 08:02PM UTC

Jobs 8

Files 140

Run time 2min

Badge

Embed ▾

pending completion

Build # 3068

Build Type

push

jenkins

Committed by Michael L Heuer

Commit Message

[ADAM-1680] Eliminate non-determinism in the ShuffleRegionJoin.

Resolves #1680. In the existing shuffle region join code, the partition
start/stop boundaries are determined by sorting the data and looking at the
coordinates of the first and last record on each partition. This is done via a
sampling process, which is fundamentally non-deterministic. Once these partition
bounds are picked, we replicate records from the right side of the join into the
partitions they overlap. There was a bug in this step that led to records being
dropped from the first partition of each contig (except for the first contig).
This was due to a bug in how the IntervalArray was being used to search for
repartitioning bounds.

This PR replaces the interval array with a map between contig names and the
partitions that contain their data. The first and last partition of each contig
are extended to the start/end of each contig.

Coverage Stats

6286 of 7562 relevant lines covered (83.13%)

6.62 hits per line

Jobs

ID	Job ID	Ran	Coverage
1	3068.1	17 Oct 2017 08:02PM UTC	82.74
2	3068.2	17 Oct 2017 08:02PM UTC	82.73
3	3068.3	17 Oct 2017 08:02PM UTC	82.74
4	3068.4	17 Oct 2017 08:03PM UTC	82.73
5	3068.5	17 Oct 2017 08:03PM UTC	82.84
6	3068.6	17 Oct 2017 08:04PM UTC	82.85
7	3068.7	17 Oct 2017 08:04PM UTC	82.85
8	3068.8	17 Oct 2017 08:04PM UTC	82.84

Source Files on build 3068

Detailed source file information is not available for this build.

bigdatagenomics / adam / 3068

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 3068

bigdatagenomics / adam / 3068

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 3068

README BADGES
x