pantsbuild / pants / 34865332339

93%

push

github

Percent-encode S3 object keys before signing the download URL (#23671)

The `s3` url handler signs a URL that differs from the URL it downloads,
so an object whose key contains a character that must be percent-encoded
— such as `+` — is unfetchable, failing with HTTP 403
`SignatureDoesNotMatch`.

Percent-encode the key once, before the URL is built, and sign the
encoded URL with `botocore.auth.S3SigV4Auth`:

```python
key = quote(unquote(request.key), safe="/~")
```

`S3SigV4Auth` signs the path verbatim (`SigV4Auth` re-encodes it and
collapses `.`/`..` segments, which are legal in a key), so the string
that is signed is the string that is downloaded. The `unquote` keeps
this idempotent for URLs that already spell the key encoded. Tests cover
both signing modes and both spellings of the key.

See the comment below for the canonical-path analysis, the verification
against a real bucket, and why `S3SigV4Auth` rather than encoding only
the downloaded URL.

---

**LLM assistance notice**
([policy](https://www.pantsbuild.org/dev/docs/contributions#llm-assistance-notice)):
the code and tests here were written primarily by Claude Code. I hit the
bug against a private S3 bucket, verified the signature mismatch and the
fix by hand against botocore 1.43.73, and reviewed and ran the tests.

7 of 7 new or added lines in 2 files covered. (100.0%)

95688 of 102998 relevant lines covered (92.9%)

3.66 hits per line