Applebot-Extended
A robots.txt token Apple introduced in 2024. Sites that block it opt out of having Applebot-collected content used to train Apple Foundation Models — while keeping Siri / Spotlight / Safari search visibility intact.
Specs
| Vendor | Apple |
| Category | DIRECTIVE |
| robots.txt token | Applebot-Extended |
| Renders JavaScript | — |
| Honors robots.txt | n/a |
User-Agent string
(not a crawler — robots.txt directive)Considerations
- Per Apple's docs: 'Applebot-Extended does not crawl webpages.' It's a control mechanism, not a fetcher.
- Only meaningful in combination with Applebot. Block only Applebot-Extended (and allow Applebot) for the search-yes-training-no outcome.
robots.txt recipe
User-agent: Applebot
Allow: /
User-agent: Applebot-Extended
Disallow: /