11790
53%

Ran 14 Jun 2019 11:17AM UTC

Jobs 1

Files 111

Run time 8s

Badge

Embed ▾

pending completion

Build # 11790

Build Type

push

travis-ci

Committed by

mrocklin

Commit Message

Cache chunk boundaries for integer slicing (#4923)

This is an alternative to #4909, to implement #4867.

Instead of caching in the class as in #4909, use functools.lru_cache.
This unfortunately has a fixed cache size rather than a cache entry
stored with each array, but simplifies the code as it is not necessary
to pass the cached value from the Array class down through the call tree
to the point of use.

A quick benchmark shows that the result for indexing a single value from
a large array is similar to that from #4909, i.e., around 10x faster for
constructing the graph.

This only applies the cache in `_slice_1d`, so should be considered a
proof-of-concept.

* Move cached_cumsum to dask/array/slicing.py

It can't go in dask/utils.py because the top level is not supposed to
depend on numpy.

* cached_cumsum: index cache by both id and hash

The underlying _cumsum is first called with _HashIdWrapper, which will
hit (very cheaply) if we've seen this tuple object before. If not, it
will call itself again without the wrapper, which will hit (but at a
higher cost for tuple.__hash__) if we've seen the same value before but
in a different tuple object.

* Apply cached_cumsum in more places

Coverage Stats

18043 of 19868 relevant lines covered (90.81%)

0.91 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
2	11790.2 (PYTHON=3.6 NUMPY=1.15.1 PANDAS=0.23.4 TEST='true' LINT='true' COVERAGE='true' PARALLEL='false' XTRATESTARGS= TEST_IMPORTS='true')	14 Jun 2019 11:18AM UTC	0	90.81	Travis Job 11790.2

Source Files on build 11790

Detailed source file information is not available for this build.

dask / dask / 11790
53%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 11790

dask / dask / 11790 53%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 11790

dask / dask / 11790
53%

README BADGES
x