snejus / beetcamp / 10238495737 / 5

91%

main: 88%

push

github

fix: use httpx instead of requests to avoid Bandcamp blocking

When using requests/urllib3, Bandcamp response to all requests with
403 errors.

Investigating why, I tried:
- using curl to send the same request: it worked
- writing a tiny Python script to `GET bandcamp.com/` with requests: it
  failed with 403
- waiting a week to see if it solved itself: no luck
- changing the above mentioned script to use http.client or httpx worked

I think that in this case, Bandcamp's Web Application Firewall (WAF)
blocks the requests based not on their contents but on an artifact of
how urllib3 builds/sends the data, since curl with exact same headers
works.

Instead of trying to identify the exact reason, which is quite hard
without any info on Bandcamp's WAF, and fix/workaround that, I rewrote
the very little required HTTP code to use httpx and sidestep the issue.

360 of 420 branches covered (85.71%)

Branch coverage included in aggregate %.

1016 of 1114 relevant lines covered (91.2%)

0.91 hits per line