If a scraper starts throwing ReadTimeout and the first instinct is to double every timeout, read the message again. It can point at the wrong place.
We made each phase of a proxied HTTPS request stall on purpose in a local lab (curl 8.5.0, requests 2.34.2, httpx 0.28.1). For a proxy that accepts the connection and never answers CONNECT, curl said "Proxy CONNECT aborted due to timeout". requests raised ReadTimeout with the target's hostname in it, though the target was never contacted. httpx said ReadTimeout too, but only after its read timer, so 8 seconds instead of 5.
The tell in requests: if the number in "(read timeout=N)" equals your connect value, the stall was in the tunnel or TLS, not on the site.
Two more from the same run. An unreachable proxy raises ProxyError, which isn't a subclass of requests.exceptions.Timeout, so code that only catches Timeout lets it crash the worker. And with no timeout set at all, curl and requests both sat there for 134 seconds, which is the Linux kernel's SYN retries and not the client.
curl -w with time_connect, http_connect, time_appconnect and time_starttransfer splits the four phases, and honestly that's the first thing I'd run.
Full table: https://proxymint.com/proxy-timeout/
Disclosure: our blog, we run a proxy service.
all 2 comments