aio-libs / yarl / 40833680725

98%

push

github

Keep an invalid percent sign as written in WHATWG mode (#2010)

<!-- Thank you for your contribution! -->

## What do these changes do?

In WHATWG mode a `%` that does not start a `%XX` escape is now kept as
written in the path, the query and the fragment of a parsed URL, as the
WHATWG URL Standard does (it is a validation error there, not a
failure). Previously yarl encoded it as `%25`, so
`http://example.com/foo%2zbar` became `http://example.com/foo%252zbar`.
Node's `URL` gives the same results as the new code.

The quoters get a `keep_invalid_pct` flag, implemented identically in
`_quoting_c.pyx` and `_quoting_py.py`, and WHATWG mode parses with three
new requoter instances that set it. RFC mode still rejects such a URL
string.

Builder methods that take decoded values (`URL.build()`, `with_path()`,
`with_query()`, `with_fragment()`, `/`, `joinpath()`, `update_query()`,
`extend_query()`) still encode `%` as `%25` in both modes: there the
value is decoded text, so `%` is a literal percent sign and must round
trip through `.path` and friends. `encoded=True` is unchanged.

In `http://example.com/foo/%2e%2` yarl still decodes `%2e` to `.`
(giving `/foo/.%2`). That has a different cause: yarl decodes escapes of
unreserved characters in both modes (RFC 3986 section 6.2.2.2
normalization), which the report treats as equivalent, so the row now
counts as following WHATWG.

## Are there changes in behavior for the user?

Yes. In WHATWG mode (the default), parsing a URL string with an invalid
`%` escape keeps the `%` instead of writing `%25`, so
`str(URL("http://h/100%"))` is now `http://h/100%`. The decoded `path`,
`query` and `fragment` are the same as before. A WHATWG URL with such a
`%` moved to RFC mode with `URL(url, mode="rfc")` keeps it as written,
as it does for other WHATWG-only characters.

## Is it a substantial burden for the maintainers to support this?

No. The quoter change is one branch in the invalid-escape path of each
implementation, cover... (continued)

99 of 198 branches covered (50.0%)

Branch coverage included in aggregate %.

151 of 151 new or added lines in 6 files covered. (100.0%)

12067 of 12220 relevant lines covered (98.75%)

0.99 hits per line