thumbor / remotecv / 39880783798

94%

push

github

Keep credentials and query strings out of logs and metric names (#121)

DetectTask logged the full image URL at INFO on every job, pyres logged the job
arguments, which hold the same URL, when it reserved, started and failed a job,
urllib echoed the URL into the exception that the failure line prints, and the
HTTP loader put the URL's netloc in four metric names. A URL with basic auth in
its userinfo or a signing token in its query string therefore ended up in the
worker log and in the Sentry event built from the failure line, and a userinfo
or port ended up inside a metric name, where ':' and '@' are statsd delimiters.

redact_url unquotes the value and accepts it with or without a scheme, since
thumbor enqueues the image URL percent-encoded and usually without one, keeps
scheme, host, port and path, and returns a placeholder instead of raising,
because pyres formats the job outside any handler while reserving it. A non-
numeric port that shows an encoded '/' in the password also yields the
placeholder, since urllib rejects that URL anyway. UniqueWorker reserves jobs
as RedactedJob, whose str() applies it to every string in the arguments,
however nested, and leaves any other payload shape alone so that reserving a
malformed job still cannot crash the worker. Metric names use the hostname
only, so the ":port" suffix disappears for origins on non-default ports and
dashboards built on those names need adjusting once. Sentry still sees the raw
URL in stack-frame variables. This change is stacked on #120 and should merge
after it.

49 of 49 new or added lines in 4 files covered. (100.0%)

601 of 675 relevant lines covered (89.04%)

4.45 hits per line