FAIRmat-NFDI / pynxtools / 23192524677

82%

push

github

Making explicit chunk cache handling policies and enabling a custom autochunker with a fall back to h5py's build-in autochunker (#741)

* Design chunking configurations for different systems

* Skeleton implementation replacement of the auto-chunking

* documentation

* Added explicit setting of rdcc config upon file creation

* Most up-to-date link in docs

* sync up with latest state from cenem feature branch, cherry-picking only those modifications pertaining to customized chunking related work, and adding the customize auto chunker code snippet from pynxtools-em and pynxtools-apm here, next step copy over the documentation from the refactoring_compression feature branch and add to that existent documentation details about what this feature branch adds as additional functionalities for customizing the chunking

* Add missing config_defaults from refactoring_compression branch

* Add documentation specific for chunking

* skeleton for tests, pulled together changes in one file

* remove unused

* remove dead code and move imports

* docstring

* fix handling of awkward cases, tests completed

* addressed remaining reviewer comment

* fix incorrect name of compression filters

* fix removing incorrect loading of logger

---------

Co-authored-by: mkuehbach <markus.kuehbach@physik.hu-berlin.de>

78 of 95 new or added lines in 4 files covered. (82.11%)

5760 of 7032 relevant lines covered (81.91%)

0.82 hits per line