Verified Confidence: high
Primary source found; it says the same thing, the scope matches, and it is reasonably current.
Safe rephrasing
Google reads only the first 500 KiB of a robots.txt file and ignores the rest.
What the source says
Google enforces a robots.txt file size limit of 500 kibibytes (KiB). Content which is after the maximum file size is ignored.
Google Search Central — How Google interprets the robots.txt specification
- Commonly cited as
- "Is there a robots.txt size limit?"
- True origin
- Google Search Central — How Google interprets the robots.txt specification
- Type
- Reference fact: A frequently needed number or rule taken directly from a primary source (e.g. Pew, Census, IAB/PwC, FTC), with its as-of date.
- Year
- Current documentation
- Data as of
- Documentation read on 2026-10-17
- Last checked
- Method & sample
- Official Google documentation (current version read Oct. 17, 2026).
- Drift between quote and source
- Google-specific limit; other crawlers may differ
- Interest
- Google operates Google Search and documents its own systems.