pantsbuild / pants / 37394615658

93%

push

github

perf(docker): Build Index over Packageable Output Paths (#23689)

## What

`infer_docker_dependencies` looped over every packageable target for
every `docker_image`, and computed each target's default output path for
each candidate file ending. The cost of this was O(images x packageable
targets). This added significant cost on repos with many images and
targets.

This adds two small rules, each computed once per run:

- `packageable_output_paths`, per file ending, maps each default output
path to the positions of the packageable targets that produce it.
- `packageable_target_positions` maps each address to its position in
`AllPackageableTargets`.

Each image then looks up its candidate paths and addresses in those
indexes and collects the matching targets in a set. The inferred
dependencies don't change: `InferredDependencies` sorts and deduplicates
them, as it did for the old loop's list.

## Measurements

| Repo | Pairs | Control mean | Candidate mean | Difference | Per-pair
differences |
|---|---|---|---|---|---|
| Large private monorepo | 3 | 132.14s | 113.03s | **-19.11s (-14.5%)**
| -17.93, -17.76, -21.63 |
| Synthetic | 5 | 69.98s | 56.00s | **-13.98s (-20.0%)** | -14.02,
-14.58, -14.27, -13.42, -13.61 |

The saving grows with images x packageable targets. The synthetic repo
matches the large private repo's counts of both (about 1,700 images and
2,750 pex binaries), and every pair on both repos was faster.

## Tests

`pants test
src/python/pants/backend/docker/util_rules/dependencies_test.py` (the
test's explicit rule list now includes the two new rules), plus
`fmt`/`lint`/`check` on the changed files.

Disclosure: investigated and implemented with claude.

30 of 30 new or added lines in 1 file covered. (100.0%)

95996 of 103309 relevant lines covered (92.92%)

3.66 hits per line