BlueBrain / libsonata / 6719124249

94%

master: 94%

Pull #298

github

Implement merge-read-extract.

Reading from parallel filesystems, e.g. GPFS, requires reading few but
large chunks. Reading multiple times from the same block/page, come with
a hefty performance penalty.

The commit implements the functionality for merging nearby reads by
adding or modifying:

  * `sortAndMerge` to allow merging ranges across gaps of a certain
    size.

  * `bulkRead` to read block-by-block and extract the requested slices
    in memory.

  * `_readSelection` to always combine reads.

  * `?fferent_edges` to optimize reading of edge IDs.

It requires a compile-time constant `SONATA_PAGESIZE` to specify the
block/pagesize to be targeted.

84 of 84 new or added lines in 3 files covered. (100.0%)

1872 of 1992 relevant lines covered (93.98%)

77.9 hits per line