ruby-grape / grape / 32581434646

97%

push

github

Tag path params as UTF-8 instead of leaving them binary (#2839)

Mustermann decodes path captures out of PATH_INFO, which Rack hands over
tagged ASCII-8BIT, and nothing re-tagged the result. Query and body params
arrive UTF-8 because Rack tags those itself, so the same value reached the
endpoint with a different encoding depending on where it came from.

That made an API's declarations disagree with themselves -- a binary string
never equals the UTF-8 literal it was written as:

    params { requires :id, type: String, values: ['café'] }

    GET /?id=café   ->  200
    GET /café       ->  400 "id does not have a valid value"

The same held for same_as, except_values and any comparison an endpoint made
against a non-ASCII literal. It also leaked into serialization: a non-ASCII
path param rendered into a JSON response drew an encoding warning from the
json gem, which that gem says will become an error in json 3.0.

Re-tag in Route#params_for, the single funnel for path-extracted values.

Nothing obliges a client to send UTF-8 -- HTTP treats the request target as
octets, and Rack's SPEC has CGI keys carry non-ASCII as ASCII-8BIT -- so
UTF-8 is the convention rather than a guarantee: it is what browsers
percent-encode with, what an IRI maps to, and what Rails settles on
(ActionDispatch::Journey::Router force_encodes every path capture to UTF-8
after unescaping it). Only the encoding changes here: the bytes are
untouched, so octets that are not UTF-8 stay invalid and are still caught
downstream instead of being silently scrubbed into something the client
never sent. Unnamed splats capture into an Array, so those are walked too.

The re-tag is unconditional and in place, hence tag_utf8!. That is safe
rather than merely cheap: Mustermann's Pattern#params builds a fresh Hash of
fresh, unfrozen strings on every call and skips its own Match cache, so the
mutation cannot escape the request. It is also what lets the Array branch
re-tag its elements with each.

S... (continued)

1223 of 1312 branches covered (93.22%)

Branch coverage included in aggregate %.

7 of 8 new or added lines in 1 file covered. (87.5%)

3842 of 3910 relevant lines covered (98.26%)

27160.5 hits per line