bodgit / rvz / 38395581020

77%

Build Type

push

github

Committed by web-flow

Commit Message

perf: Compress in parallel (#191)

* perf: Compress groups in parallel

Prepare groups in jobs of up to 2 MiB of raw chunks or one 2 MiB block
of partition data, with up to GOMAXPROCS jobs running ahead of a single
writer that stores the results in order. The writer still assigns
offsets, reuses groups and pads, so the output is identical to
compressing sequentially.

The Zstandard encoder is no longer limited to one concurrent encode,
which serialised every job behind a single encoder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf: Find padding seeds faster

Seed only converts the block of words it needs to recover a seed rather
than the rest of the sector, and counts how many bytes the seed
generates a block at a time instead of a byte at a time. On a Wii
partition, which is mostly padding, this was a fifth of the CPU time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf: Always entropy code literals when compressing

At level 5 the Zstandard encoder skips entropy coding literals when it
finds few matches, whereas libzstd, and so Dolphin, always does. Disc
images have a lot of audio and video like that, which was being stored
uncompressed. This makes the GameCube test image 32 MB smaller and the
Wii test images up to 14 MB smaller, for about 0.2-0.4 seconds more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Coverage Stats

180 of 204 new or added lines in 4 files covered. (88.24%)

4 existing lines in 2 files now uncovered.

1469 of 1912 relevant lines covered (76.83%)

0.87 hits per line