Google-Extended
A robots.txt token Google introduced in September 2023. Sites that include Disallow: / for it opt out of having Googlebot-fetched content used to train Gemini, Vertex AI, and Google's broader generative AI stack — while keeping their Google Search visibility.
Specs
| Vendor | |
| Category | DIRECTIVE |
| robots.txt token | Google-Extended |
| Renders JavaScript | — |
| Honors robots.txt | n/a |
User-Agent string
(not a crawler — robots.txt directive)Considerations
- Not an active crawler — you will never see Google-Extended/1.0 in your logs.
- Only meaningful in combination with Googlebot. Blocking only Google-Extended (while keeping Googlebot allowed) produces the intended search-yes-training-no outcome.
robots.txt recipe
User-agent: Googlebot
Allow: /
User-agent: Google-Extended
Disallow: /