aio-libs / yarl / 40693730913

98%

push

github

Keep quotes, backticks and braces in WHATWG special hosts (#2008)

<!-- Thank you for your contribution! -->

## What do these changes do?

In WHATWG mode the host of a special URL (`http`, `https`, `ws`, `wss`,
`ftp`, `file`) keeps `"`, a backtick, `{` and `}`, as the WHATWG URL
Standard only rejects its forbidden domain code points there.
`http://!"$&'()*+,-.;=_` + "`{}~/" now parses as written, as Node and
browsers do. The change applies to parsing, `URL.build(host=,
authority=)`, `with_host()`, `with_scheme()` and `join()`, and to
percent-decoded hosts (`http://ho%7Bst/` is `http://ho{st/`).

`idna` rejects these characters, so a non-ASCII host with them is mapped
with UTS #46 (non-transitional, without the STD3 rules, as WHATWG does),
a label with them is encoded with Punycode and the other labels with
IDNA as before: `http://faß.a{b}/` is `http://xn--fa-hia.a{b}/` and
`http://faß{b}/` is `http://xn--fa{b}-mqa/`, not `fass` from the
IDNA2003 fallback. `URL.host` decodes such labels back. RFC 3986 mode
still rejects all of these hosts.

## Are there changes in behavior for the user?

Yes. In the default WHATWG mode, hosts with `"`, a backtick, `{` or `}`
are accepted and lowercased instead of raising `ValueError`. The
forbidden domain code points (space, `%`, `^`, `|`, `<`, `>`, `[`, `]`,
`\`, controls, and so on) are still rejected. RFC mode is unchanged.

## Is it a substantial burden for the maintainers to support this?

No. The new check only runs when the reg-name check already failed, so
valid hosts take the same path as before.

## Related issue number

N/A

## Checklist

- [x] I think the code is well written
- [x] Unit tests for the changes exist
- [x] Documentation reflects the changes
- [ ] If you provide code modification, please add yourself to
`CONTRIBUTORS.txt`
- [x] Add a new news fragment into the `CHANGES/` folder

Drafted with Claude Code (Claude Opus 5.5); reviewed by @asvetlov.

<details>
<summary>Agent run details (optional, ... (continued)

97 of 194 branches covered (50.0%)

Branch coverage included in aggregate %.

157 of 157 new or added lines in 4 files covered. (100.0%)

11877 of 12027 relevant lines covered (98.75%)

0.99 hits per line