AI platforms crawl far more than they refer: Anthropic made about 70,900 page requests per referral (Cloudflare)
Verified Confidence: medium
Primary source found; it says the same thing, the scope matches, and it is reasonably current.
- Commonly cited as
- "AI bots crawl thousands of pages for every visitor they send"
- True origin
- Cloudflare Blog — The crawl before the fall… of referrals: understanding AI's impact on content providers (2025)
- Type
- Stat check: A widely repeated statistic traced back to its origin.
- Year
- 2025
- Data as of
- Example period June 19–26, 2025
- Last checked
- What the source says
the ratios range from Anthropic's 70,900:1 down to Mistral's 0.1:1. This means that Anthropic's AI platform Claude made nearly 71,000 HTML page requests for every HTML page referral
- Method & sample
- Cloudflare network data: HTML page requests by each platform's crawlers vs referrals from its hostnames, normalized to one referral.
- Drift between quote and source
- Date: a one-week example; ratios change over time
- Scope: Cloudflare's network only
- Interest
- Cloudflare sells bot management and pay-per-crawl tools.
Safe rephrasing
Cloudflare's data for June 19–26, 2025 showed crawl-to-referral ratios ranging from about 70,900:1 for Anthropic down to 0.1:1 for Mistral; ratios vary week to week.
Sources we opened
Machine-readable: /api/claims/anthropic-70900-crawls-per-referral-cloudflare.json · Last checked 2026-10-16