thumbor / remotecv / 39887600581

94%

push

github

Send a configurable User-Agent when fetching images (#128)

The HTTP loader fetched images with urllib's default Python-urllib/3.x
User-Agent, which the reporter of #87 saw challenged by Cloudflare's Browser
Integrity Check while thumbor's own requests to the same origin went through.
The new --loader-user-agent option (IMAGE_LOADER_USER_AGENT) sets the header
and defaults to remotecv/<version>, mirroring thumbor's
HTTP_LOADER_DEFAULT_USER_AGENT.

The default changes what every deployment sends, and Cloudflare documents
challenging any non-standard User-Agent, so remotecv/<version> is not
guaranteed to pass either; operators behind such a check can set the value
thumbor uses or add a WAF skip rule. The value must be non-empty printable
ASCII and is checked at startup: http.client rejects a header with CR/LF or
characters outside Latin-1 on every request, which the loader reports as an
invalid image URL, and an empty header is the missing User-Agent case that
Cloudflare challenges.

11 of 11 new or added lines in 2 files covered. (100.0%)

638 of 679 relevant lines covered (93.96%)

4.7 hits per line