1
72%
master: 72%

Ran 21 Aug 2025 10:21AM UTC

Files 1952

Run time 1min

Badge

Embed ▾

Committed 21 Aug 2025 09:22AM UTC coverage: 71.162% (+0.003%) from 71.159%

Job # 17122751530.1

Build Type

push

github

Committed by

web-flow

Commit Message

VSI archive (/vsizip/, /vsiar/) performance improvements with large number of files (#12939)

Traversal of large archives (many entries, especially in a large
hierarchy) can be slow.

This change massively improves performance of random lookups by path
within an archive, as well as traversal using VSIReadDirRecursive. The
main change is to avoid full traversal of the VSIArchiveContent::entries
array for each path lookup.

I achieved this by building an index mapping each directory entry
to the indices of its children in the entries list, and using that index
to speed up lookups by path.

This also has a massive perf improvement for
VSIReadDirRecursive since it works by calling ReadDirEx for each
subdirectory (which previously meant looking at all entries). Now it
uses the directory index to immediately jump to where the directory is
in the entries list, avoiding visiting other entries.

On my laptop, ReadDirRecursive on a zip file containing 600 dirs each
containing 600 files:

* previous master gdal: 728 seconds
* after this change: 4.2 seconds

Run Details

578524 of 812970 relevant lines covered (71.16%)

286750.1 hits per line

OSGeo / gdal / 17122751530 / 1
72%
master: 72%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 17122751530.1

OSGeo / gdal / 17122751530 / 1 72% master: 72%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 17122751530.1

OSGeo / gdal / 17122751530 / 1
72%
master: 72%

README BADGES
x