Claude-User
Fetches URLs in real time when a Claude user asks a question that requires current web data, or when Claude's agent / web-search tool follows a link. Per-query, user-driven.
Specs
| Vendor | Anthropic |
| Category | FETCH |
| robots.txt token | Claude-User |
| Renders JavaScript | RENDERS JS |
| Honors robots.txt | yes |
| Reverse-DNS pattern | *.anthropic.com |
| IP-range source | https://claude.com/crawling/bots.json |
User-Agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +https://anthropic.com/claude-user)Considerations
- Distinct from ClaudeBot — site can block the training token but allow Claude-User to participate in Claude's live answers.
- Uses headless browser infrastructure when the task requires it (agent tool calls, 'browse a page').
- Bursty traffic profile — small volume, high relevance per request.
robots.txt recipe
User-agent: Claude-User
Allow: /Sources: Anthropic · Crawler FAQ