ukeeper / ukeeper-readability / 28681422902

78%

master: 78%

Build Type

Pull #83

github

Committed by paskal

Commit Message

Block SSRF to non-public addresses via connect-time IP guard

POST /api/extract is unauthenticated and the extractor fetched any URL it
was given with no host validation, so a caller could reach
http://169.254.169.254/... (cloud metadata), localhost and other internal
hosts; image extraction amplified it by following <img src> into more
internal fetches.

The page retriever and image fetcher reject targets that resolve to
loopback, private, link-local, unspecified, multicast and other IANA
special-purpose ranges (0.0.0.0/8, CGNAT, benchmarking, documentation,
reserved, broadcast) via a net.Dialer Control hook that checks the actual
IP at connect time, so it also covers redirect-to-internal and DNS
rebinding; the guarded transport disables env proxies so validation can't
be bypassed through a proxy. extractWithRules additionally validates the
resolved host up front, so the Cloudflare path (which fetches remotely and
isn't covered by the dialer) enforces the same policy.

Image probes share a single lazily-built client (and its guarded transport)
rather than constructing one per <img>, so enabling the guard by default
doesn't churn transports/FDs.

Enabled by default; operators opt out with --allow-private-networks for
trusted intranet extraction. The default HTTPRetriever inherits the flag
from UReadability so package-level callers are guarded too.

Pull Request Pull Request #83: Block SSRF to non-public addresses via connect-time IP guard

Coverage Stats

79 of 99 new or added lines in 5 files covered. (79.8%)

13 existing lines in 2 files now uncovered.

832 of 1069 relevant lines covered (77.83%)

9.26 hits per line