ooni / probe-cli / 6198965727

72%

Build Type

push

github

Committed by web-flow

Commit Message

feat(testingx): introduce more comprehensive HTTP(S) proxy (#1274)

We want more comprehensive testing of how we use proxies during the
bootstrap. Tests should encompass both SOCKS5 and HTTP(S) proxies. Tests
should support using both the host network and using netem.

This diff starts paving the way for improving proxy testing, by
introducing in `./internal/testingx/httpproxy.go` an HTTP(S) proxy
implementation supporting both the case of HTTP over HTTP(S) proxy
(where you use the host header) and HTTPS over HTTP(S) proxy (where the
client issues a `CONNECT` request to the remote endpoint).

The previous implementation (in `./internal/testingx/httptestx.go`) is
still there, for now. We used the previous implementation, which only
supported the host header, as a starting point for writing the new one.

More in detail, this diff introduces the new proxy and its tests.
Because testing the proxy functionality is a bit complex, I chose to use
a separate package and also write tests for the tests. Obviously, we're
still using `netxlite` as the underlying library, so there is some
circularity in testing, but `netxlite` also has its own tests, so we
should probably be fine.

The separate package with tests is `./internal/testingproxy`.

While working on this package, I realized the need to forward the CA
used by the proxy. This happens in
`./internal/testingproxy/hosthttps.go`. If we do not do this, we see the
following failure:

```
=== RUN   TestWorkingAsIntended/fetching_https://www.example.com/_using_the_host_network_and_an_HTTPS_proxy
2023/09/15 15:28:39 debug > GET https://www.example.com/
2023/09/15 15:28:39 debug >                        

2023/09/15 15:28:39 dialerWithAssertions: verified that the network is tcp as expected
2023/09/15 15:28:39 dialerWithAssertions: verified that the address is 127.0.0.1 as expected

2023/09/15 15:28:39 debug dial 127.0.0.1:61772/tcp...
2023/09/15 15:28:39 debug dial_address 127.0... (continued)

Run Details

36 of 216 new or added lines in 7 files covered. (16.67%)

8 existing lines in 2 files now uncovered.

22438 of 27856 relevant lines covered (80.55%)

27.29 hits per line