ClaudeBot

What is ClaudeBot?

ClaudeBot is Anthropic's crawler, the company behind Claude, and it's controlled from robots.txt. Anthropic also declares two other names: Claude-User, which visits a site when someone asks Claude about it, and anthropic-ai, an older name the company still honors. All three are treated separately, so a rule aimed at one doesn't cover the others. A common case is an old anthropic-ai block nobody remembers writing that's still sitting there.

Related terms

robots.txt

The file that tells crawlers which parts of your site they can access.

GPTBot

OpenAI's crawler. Not the only name the company declares.

PerplexityBot

Perplexity AI's crawler, with a second agent for answering live.

And what about your brand?

We’ll tell you how many times your brand shows up in ChatGPT, Gemini, Perplexity, and Claude when someone asks about your category, and who shows up in your place. Free, no card required.

I want my free diagnosis Check my site’s access