pantsbuild / pants / 35236

0%

main: 93%

push

travis-ci-com

Formalize support for VCS-style requirements via PEP 440 (#10728)

Since 2016, users have been asking for a way to install wheels from a URL, e.g. from Git.

There are two approaches to VCS-style requirements:

1) Pip's proprietary syntax, e.g. `git+https://github.com/pypa/pip.git#egg=pip`. This is what most people are familiar with. https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support
2) PEP 440's standardized "direct references", e.g. `pip@ git+https://github.com/pypa/pip.git`.

Turns out, ever since we upgraded to Pex 2.0, approach #2 has worked to achieve this goal. But few users knew about it (including us), and approach #1 has continued to not work.

We use setuptools to parse our requirements before passing to Pex. Why? We need to normalize it to, for example, strip comments from `requirements.txt`; and because we need the parsed information to do things like create a module mapping for dependency inference. However, `pkg_resources.Requirement.parse()` chokes on Pip's VCS style, and only understands PEP 440.

Both the Pip VCS and PEP 440 approaches achieve the same end goal. We could use some custom libraries like https://github.com/sarugaku/requirementslib and https://github.com/davidfischer/requirements-parser to allow for Pip's proprietary format, but that adds new complexity to our project.

Nevertheless, few users are familiar with PEP 440, so we eagerly detect if they're trying to use the Pip style and have a nice error message for how to instead use PEP 440:

```
▶ ./pants list 3rdparty/python: --no-print-exception-stacktrace
13:25:59.67 [ERROR] 1 Exception encountered:

  MappingError: Failed to parse 3rdparty/python/BUILD:
Invalid requirement 'git+https://github.com/django/django.git#egg=django' in 3rdparty/python/requirements.txt at line 32: Parse error at "'+https:/'": Expected stringEnd

It looks like you're trying to use a pip VCS-style requirement?
Instead, use a direct reference (PEP 440).

In... (continued)

18 of 18 new or added lines in 5 files covered. (100.0%)

0 of 29102 relevant lines covered (0.0%)

0.0 hits per line