pantsbuild / pants / 18727405282

80%

Build Type

push

github

Committed by web-flow

Commit Message

Enable scoping indexes for specific projects/platforms when generating lockfiles (#22760)

### Problem
The current `generate-lockfiles` goal only allows one locked version for
each requirement. This leads to issues when using certain libraries like
Pytorch and Tensorflow which have some complexities in the way the
wheels are published and tagged. See
https://github.com/pantsbuild/pants/issues/18965

An example with pytorch: the Linux wheels published to PyPI include cuda
dependencies, while the macos ones do not. There is a `+cpu` tag in the
pytorch-hosted index, _but only for Linux_. The current setup means that
we can't lock a cpu-only version of pytorch that works on both macos and
linux.

### Proposed solution
As of [pex 2.56.0](https://github.com/pex-tool/pex/releases/tag/v2.56.0)
it is possible to create a universal lock with two locked resolves (one
for macos, one for linux) by adding an appropriately scoped index:
```bash
pex3 lock create \
    --style universal \
    --target-system linux \
    --target-system mac \
    --elide-unused-requires-dist \
    --interpreter-constraint "CPython==3.13.*" \
    --index pytorch=https://download.pytorch.org/whl/cpu \
    --source "pytorch=torch; sys_platform != 'darwin'" \
    --indent 2 \
    -o lock.json \
    torch
```
This PR adds the option of specifying sources through a new
`[python.resolves_to_sources]` option. Named indexes can already be
defined in `[python-repos.indexes]`

Run Details

94 of 96 new or added lines in 6 files covered. (97.92%)

3 existing lines in 1 file now uncovered.

77860 of 96990 relevant lines covered (80.28%)

3.08 hits per line