ACR · 248 crawlers
AI Bot Directory
Reference guides for AI and search crawlers in the
AI Crawler Registry (ACR).
Learn what each bot does and how to allow or block it in robots.txt.
Free-check your site against all 248 bots →
GPTBot
GPTBot
OpenAI
robots.txt guide →
ClaudeBot
ClaudeBot
Anthropic
robots.txt guide →
Google-Extended
Google-Extended
robots.txt guide →
PerplexityBot
PerplexityBot
Perplexity
robots.txt guide →
Bytespider
Bytespider
ByteDance
robots.txt guide →
CCBot
CCBot
Common Crawl
robots.txt guide →
GoogleBot
Googlebot
robots.txt guide →
BingBot
Bingbot
Microsoft
robots.txt guide →
anthropic-ai
anthropic-ai
Anthropic
robots.txt guide →
Amazonbot
Amazonbot
Amazon
robots.txt guide →
Full registry
This directory launches with 10 high-impact bots. The complete ACR registry documents 248 crawlers — AI, search, SEO, social, ads, security, and more.