aio-libs / yarl / 40473362404

98%

push

github

Parse the host of a non-special URL as an opaque host (#1999)

<!-- Thank you for your contribution! -->

## What do these changes do?

In the default WHATWG mode the host of a URL whose scheme is not special
is now parsed as a WHATWG *opaque host*: it is not lowercased or encoded
with IDNA, C0 controls and non-ASCII code points are percent-encoded,
and only the forbidden host code points are rejected (a lone `%`, `"` or
`{` are allowed). Parsing, `URL.build(host=..., authority=...)`,
`with_host()` and `with_scheme()` follow; moving such a host to a
special scheme still lowercases it and encodes it as a domain.
`URL.host` percent-decodes only the non-ASCII characters of an opaque
host, so it can be passed back to `with_host()`. IPv6 addresses in
brackets, RFC 3986 mode, special schemes, relative references and
`encoded=True` are unchanged.

In both modes a non-special URL with an authority now keeps its empty
path before a query or a fragment, so `sc://h?x` is no longer written as
`sc://h/?x`, matching what `sc://h` already did. Special schemes and
relative references keep the `/`. Equality and hashing are unchanged: an
empty path still compares equal to `/`, as `sc://h == sc://h/` already
did.

Conformance report: 11 rows move into *yarl follows each standard in its
mode* (7 from *WHATWG mode differs*: `sc://faß.ExAmPlE/`,
`sc://ñ.test/`, `sc://ñ`, `x` against `sc://ñ`, `sc://%/`, the C0
control host and `` foo://!"$%&'()*+,-.;=_`{}~/ ``; 4 from *Both modes
differ*: `sc://ñ?x`, `sc://ñ#x`, `?x` and `#x` against `sc://ñ`). The
*yarl differs, RFC 3986+3987 and WHATWG agree* section stays empty.

## Are there changes in behavior for the user?

Yes, for non-special schemes only. In WHATWG mode `URL("sc://ExAmPlE/")`
keeps its case and `URL("sc://ñ")` is `sc://%C3%B1` instead of
`sc://xn--ida`, so `URL("redis://Host")` no longer equals
`URL("redis://host")`. In both modes `str(URL("sc://h?x"))` is
`sc://h?x` instead of `sc://h/?x`; the same applies to `hu... (continued)

97 of 194 branches covered (50.0%)

Branch coverage included in aggregate %.

231 of 231 new or added lines in 2 files covered. (100.0%)

11523 of 11673 relevant lines covered (98.71%)

0.99 hits per line