ScrapingEvals

Crawl many pages

Frameworks for queues, discovery, retries, and larger jobs. A single-page access score is not a crawl benchmark.

3 projects · Results appear only where a public run is available.

Crawlee
Tested 2026-09-29

Crawl pages with queues, retries and browser options. The current test measures one-page CheerioCrawler HTTP downloads, not crawling scale.

Recent field test
Scrapy
Tested 2026-09-29

Crawl sites in Python with queues, parsing and pipelines. The current test measures single-page HTTP downloads, not crawling scale.

Recent field test
spider-rs

Crawl many URLs with a Rust-based tool. Crawl coverage has not been tested here.

Listed, not tested

Need another kind of tool? See the full inventory.