mrocklin / dask / 949

94%

master: 94%

Build Type

push

travis-ci

Committed by Matthew Rocklin

Commit Message

Write dask array to a stack of .npy files

Fixes #684

This partitions the dask.array along one axis and stores each block along
that axis as a single .npy file in the specified directory

Example
-------

>>> x = da.ones((5, 10, 10), chunks=(2, 4, 4))  # doctest: +SKIP
>>> da.to_npy_stack('data/', x, axis=0)  # doctest: +SKIP

$ tree data/
data/
|-- 0.npy
|-- 1.npy
|-- 2.npy
|-- info

The ``.npy`` files store numpy arrays for ``x[0:2], x[2:4], and x[4:5]``
respectively, as is specified by the chunk size along the zeroth axis.  The
info file stores the dtype, chunks, and axis information of the array.

You can load these stacks with the ``da.from_npy_stack`` function.

>>> y = da.from_npy_stack('data/')  # doctest: +SKIP

Run Details

25 of 25 new or added lines in 1 file covered. (100.0%)

7125 of 7574 relevant lines covered (94.07%)

0.94 hits per line