aio-libs / yarl / 36565635998

98%

push

github

Parse all-numeric special-scheme hosts as IPv4 in WHATWG mode (#1940)

<!-- Thank you for your contribution! -->

## What do these changes do?

For the special schemes, the WHATWG URL Standard parses a host that ends
in a number as an IPv4 address. #1933 followed that only for hosts with
a name label before the number (`foo.123` is rejected) and kept hosts
made only of numbers as written. In the default WHATWG mode they now go
through the WHATWG IPv4 parser too: `0x7f.1` becomes `127.0.0.1`, `256`
becomes `0.0.1.0`, `1.2.3.4.` becomes `1.2.3.4`, and `256.0.0.1`,
`1.2.3.4.5`, `09.1.1.1` or `4294967296` are rejected with `ValueError`.
This applies to `URL()`, `URL.build()` (`host=` and `authority=`),
`with_host()`, and to `with_scheme()`, `join()` with a `//host`
reference and a mode change, which can move a host into a special-scheme
WHATWG URL. RFC 3986 mode keeps such hosts as a reg-name.
Percent-encoded hosts are still left alone.

The conformance report moves 38 cases from "WHATWG mode differs from
WHATWG, RFC mode follows RFC 3986" to "yarl follows each standard in its
mode", and a fullwidth-digit IDNA host (`http://０Ｘｃ０．０２５０．０１`) from
"Both modes differ" to "RFC mode differs, WHATWG mode follows WHATWG".
Nothing enters "yarl differs, RFC 3986 and WHATWG agree" (still only
`http://[::%31]`), and "agree" stays at 450.

## Are there changes in behavior for the user?

Yes, hence the `breaking` fragment: all-numeric special-scheme hosts are
normalized or rejected in the default mode. Other schemes and
`mode="rfc"` are unchanged.

## Is it a substantial burden for the maintainers to support this?

No. The WHATWG host is computed inside the cached `_encode_host()`,
which now returns it in place of the #1933 flag; callers compare it by
identity with the encoded host, so the common case costs one `is not`
check.

## Related issue number

Follow-up to #1933. Needs aio-libs/aiohttp#13861 (and its backports to
3.14 and 3.15) for the aiohttp CI jobs, since ai... (continued)

97 of 194 branches covered (50.0%)

Branch coverage included in aggregate %.

171 of 172 new or added lines in 3 files covered. (99.42%)

9065 of 9189 relevant lines covered (98.65%)

0.99 hits per line