Applebot-Extended
| Vendor | Apple |
| Type | AI training opt-out token (not a separate crawler) |
| robots.txt token | Applebot-Extended |
| JavaScript rendering | n/a — control directive only |
| Honors robots.txt | Yes |
| Vendor docs | support.apple.com/HT204683 |
What it actually is
Applebot-Extended is not a separate bot. It is a robots.txt token that Apple introduced in 2024 to let site owners opt out of having Applebot's collected content used to train Apple Foundation Models (the on-device LLMs powering Apple Intelligence), while still allowing Applebot to crawl for Siri / Spotlight / Safari search.
The HTTP request from Applebot remains a single fetch with a single UA (Applebot/0.1). The robots.txt directive determines what Apple does with the data on its end.
How to use
To opt out of Apple Intelligence training but keep Siri/Spotlight visibility:
User-agent: Applebot
Allow: /
User-agent: Applebot-Extended
Disallow: /To opt out of both (rare — also removes Siri visibility):
User-agent: Applebot
Disallow: /Why it exists
When Apple introduced Apple Intelligence in 2024 it needed a way for publishers to opt out of model training without losing search visibility. Rather than a second crawler, Apple used the robots.txt mechanism to express the policy. Google later adopted the same pattern with Google-Extended.
Quirks
- Not an active crawler. You will never see an
Applebot-Extended/1.0
UA in your logs.
- Only meaningful in combination with
Applebot— blocking only
Applebot-Extended while allowing Applebot produces the intended outcome (search yes, training no).