omry / omegaconf / 797ba0e6-7796-4a72-bbdc-38bf5b1d9ce4

100%

master: 100%

push

circleci

Allow backslash-escaping of special characters in key paths

Keys in YAML can legally contain dots, brackets, and equals signs, but there
was previously no way to target them via OmegaConf.update(), OmegaConf.select(),
OmegaConf.from_dotlist(), or OmegaConf.from_cli(). All of those converge on
split_key() to parse the key path, and the parser had no concept of escaping.

This change adds backslash escaping so those keys are reachable:

    cfg = OmegaConf.create({"a.b": 42})
    OmegaConf.select(cfg, r"a\.b")    # 42
    OmegaConf.update(cfg, r"a\.b", 1)

    cfg = OmegaConf.from_dotlist([r"a\.b\=c=42"])
    assert cfg["a.b=c"] == 42

Escape rules (backslash is only special before . [ ] =):
    \.  →  literal dot in key
    \[  →  literal [ in key
    \]  →  literal ] in key
    \=  →  literal = in key (most relevant for dotlist / CLI keys)
    \x  →  \x unchanged (backslash passes through before any other character)

Breaking change: a backslash immediately before one of the delimiter characters
(. [ ] =) now escapes that delimiter instead of being treated as a literal
backslash followed by an active delimiter. Concretely, a key path like r"a\.b"
previously resolved to the key "a\" nested under the root followed by key "b";
it now resolves to the single key "a.b". Code that relied on the old behavior
for such paths will need to be updated.

Note for CLI users: shells consume single backslashes, so quote the argument
or double-escape: 'a\.b=1' or a\\.b=1.

Closes #1230.

---

Implementation

split_key() previously used two compiled regexes applied sequentially (dot-split,
then bracket-split). With escaping in play that approach breaks — a delimiter
character inside a key would be indistinguishable from a real delimiter — so the
regexes are replaced with a small character-by-character parser (dot mode /
bracket mode) that applies escape rules in both modes.

A fast path checks for a backslash before entering the parser, so key paths
without escaping ne... (continued)

64 of 64 new or added lines in 2 files covered. (100.0%)

3667 of 3667 relevant lines covered (100.0%)

1.0 hits per line