HTTPX
Modern sync/async HTTP for Python, same wall as Requests.
HTTPX is the modern Python HTTP client — sync and async APIs, HTTP/2 support, sane defaults. We ran httpx 0.28.1 (httpx.Client.get, redirects on, cookies kept) against the 21-site public corpus with no proxy.
It passed 3 of 21: Amazon search, Target search, and Rightmove. Amazon is the interesting one — plain Requests got a 503 on the same target minutes earlier, so small differences in default headers and protocol negotiation do move the needle at the margin. Everything else failed the same way it failed for Requests: 403s from eBay, Tripadvisor, Zillow, Indeed, Yelp, Trustpilot, Craigslist, and Wikipedia, 429s from Expedia and Realtor and Yahoo, robot checks from Walmart, AliExpress, Airbnb, CoinMarketCap, Reddit, and YouTube.
Like Requests, it fails fast and honestly — most refusals in well under half a second, with clean status codes.
Test conditions
21 real public sites (public-tough-v1), 2026-05-24, no proxy, single residential IP, Linux. Deterministic validators: expected status code plus required and forbidden content markers. Pass rates measure the tool and the network position together. Without a proxy, results reflect how each site's protections treat one plain residential connection as much as the tool itself.
What worked
- +Amazon search passed where plain Requests drew a 503 — default header/protocol differences are measurable.
- +Target and Rightmove passed with zero configuration.
- +Clean, fast failure signals; the async API makes it the natural high-throughput baseline.
What failed
- −18 of 21 targets blocked with the same 403/429/captcha wall that stops every plain HTTP client.
- −Wikipedia 403s the default User-Agent, same as Requests.
- −No TLS impersonation — protected sites identify it immediately.
When to use it
The default pick for new Python scraping code that talks to APIs and unprotected pages, especially when you want async. Against bot protection it needs the same help Requests does.
A better Requests for new code; identical odds against real protection.
Run evidence
Every attempt below is backed by saved artifacts: request/response metadata, body content, and (for browsers) screenshots. Raw JSON and artifacts live in the runner data directory.
Run data not available in this build.