BlueBrain / libsonata / 6769244473

94%

push

github

Optimize loading `?fferent_edges`. (#298)

* Implement merge-read-extract.

Reading from parallel filesystems, e.g. GPFS, requires reading few but
large chunks. Reading multiple times from the same block/page, come with
a hefty performance penalty.

The commit implements the functionality for merging nearby reads by
adding or modifying:

  * `sortAndMerge` to allow merging ranges across gaps of a certain
    size.

  * `bulkRead` to read block-by-block and extract the requested slices
    in memory.

  * `_readSelection` to always combine reads.

  * `?fferent_edges` to optimize reading of edge IDs.

It requires a compile-time constant `SONATA_PAGESIZE` to specify the
block/pagesize to be targeted.

* Move include.

82 of 84 new or added lines in 3 files covered. (97.62%)

1868 of 1988 relevant lines covered (93.96%)

77.83 hits per line