ukeeper / ukeeper-readability / 28705617872

78%

master: 78%

Pull #83

github

Block SSRF to non-public addresses via connect-time IP guard

POST /api/extract is unauthenticated and the extractor fetched any URL it
was given with no host validation, so a caller could reach
http://169.254.169.254/... (cloud metadata), localhost and other internal
hosts; image extraction amplified it by following <img src> into more
internal fetches.

The page retriever and image fetcher reject targets that resolve to
loopback, private, link-local, unspecified, multicast and other IANA
special-purpose ranges (0.0.0.0/8, CGNAT, benchmarking, documentation,
reserved, broadcast) via a net.Dialer Control hook that checks the actual
IP at connect time, so it also covers redirect-to-internal and DNS
rebinding; the guarded transport disables env proxies so validation can't
be bypassed through a proxy. extractWithRules additionally validates the
resolved host up front, so the Cloudflare path (which fetches remotely and
isn't covered by the dialer) enforces the same policy.

Image probes share a single lazily-built client (and its guarded transport)
rather than constructing one per <img>, so enabling the guard by default
doesn't churn transports/FDs; the probe request no longer forces
Connection: close, letting the shared client reuse connections.

Enabled by default; operators opt out with --allow-private-networks for
trusted intranet extraction. The default HTTPRetriever inherits the flag
from UReadability so package-level callers are guarded too.

81 of 101 new or added lines in 5 files covered. (80.2%)

833 of 1070 relevant lines covered (77.85%)

9.26 hits per line