SEARCH · OpenAI

OAI-SearchBot

ChatGPT Search index

Builds OpenAI's search index — the corpus ChatGPT Search consults when a user asks a question that needs current web data. Pages indexed by it are eligible to appear as cited sources.

Specs

VendorOpenAI
CategorySEARCH
robots.txt tokenOAI-SearchBot
Renders JavaScriptHTTP only
Honors robots.txtyes
Reverse-DNS pattern*.openai.com
IP-range sourcehttps://openai.com/searchbot.json

User-Agent string

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot

Considerations

  • Distinct from GPTBot (training) and ChatGPT-User (live retrieval). The robots.txt directive should be set independently per token.
  • Sites that want to appear in ChatGPT Search should explicitly allow this UA.
  • HTTP-only — SSR is required for content visibility.

robots.txt recipe

User-agent: OAI-SearchBot
Allow: /
Related bots: GPTBot · ChatGPT-User · Googlebot

Sources: OpenAI · Crawler overview · OpenAI · SearchBot announcement (Aug 2024)

← Back to directory