11356
53%

Ran 27 Apr 2019 02:43PM UTC

Jobs 1

Files 108

Run time 6s

Badge

Embed ▾

pending completion

Build # 11356

Build Type

push

travis-ci

Committed by

mrocklin

Commit Message

Change cum-aggregation last-nonnull-value algorithm (#4736)

Previously we would use groupby.last in order to get the last non-null value of a partition.  This was suboptimal in two ways:

-  Less advanced dataframe libraries, like cudf, might not implement groupby-last, and so couldn't use this algorithm
-  It was somewhat expensive
-  It failed if there were empty partitions

Our new approach tries two paths:

-  First, use a normal Python for loop and `.iloc` on the last ten rows of a column.  This is faster than the groupby.last call and only uses iloc, which is somewhat more likely to be around.
-  If that didn't work (there are lots of nulls) then use `s[s.notna()][-1]`, which does a full scan, but is doesn't iterate with Python

Run Details

17192 of 18966 relevant lines covered (90.65%)

0.91 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
3	11356.3 (PYTHON=3.6 NUMPY=1.15.1 PANDAS=0.23.4 TEST='true' LINT='true' COVERAGE='true' PARALLEL='false' XTRATESTARGS= TEST_IMPORTS='true')	27 Apr 2019 02:43PM UTC	0	90.65	Travis Job 11356.3

Source Files on build 11356

Detailed source file information is not available for this build.

dask / dask / 11356
53%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 11356

dask / dask / 11356 53%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 11356

dask / dask / 11356
53%

README BADGES
x