Ouranosinc / xclim / 11275206436

92%

push

github

Block-mapped resample with the help of flox (#1848)

<!--Please ensure the PR fulfills the following requirements! -->
<!-- If this is your first PR, make sure to add your details to the
AUTHORS.rst! -->
### Pull Request Checklist:
- [ ] This PR addresses an already opened issue (for bug fixes /
features)
    - This PR fixes #xyz
- [x] Tests for the changes have been added (for bug fixes / features)
- [x] (If applicable) Documentation has been added / updated (for bug
fixes / features)
- [x] CHANGELOG.rst has been updated (with summary of main changes)
- [x] Link to issue (:issue:`number`) and pull request (:pull:`number`)
has been added

### What kind of change does this PR introduce?

Implements `resample_map`. This function is meant for all
`da.resample(...).map(...)` calls. These, `flox` cannot improve
automatically so we use some flox logic to help. The idea is to map the
resample-map construct on each block in parallel. This is possible by
first rechunking the array so that chunks boundary fit with resampling
period boundaries (this is a flox function).

The main improvement should come from the fact that `map_blocks` hides
much of the complexity to `dask`, so the resulting graph is much
lighter. I still have to better test the performance of this. My goal
would be to have some short text in xclim's doc that highlights when the
option is useful and when it is not. The option is activated through
`set_options`.

The current function works only when the input object is of the same
type as the output one. So some functions couldn't be wrapped with this
yet. The most important untouched code for the moment is the missing
checks where I think this could help a lot.

### Does this PR introduce a breaking change?
It should not. This is completely optional.

### Other information:
In progress, I still need to prove the performance boost.

This depends on #1845 because I need all improvements for PC.

65 of 71 new or added lines in 8 files covered. (91.55%)

9288 of 10380 relevant lines covered (89.48%)

7.56 hits per line