OAI-SearchBot

OpenAI · Search-index crawler

OAI-SearchBot is OpenAI’s search-index crawler — it crawls proactively to build the search index the assistant cites in answers. Allowing it is how your pages become citable.

User-agent string

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot

Source: vendor-documented at developers.openai.com/api/docs/bots.

Does OAI-SearchBot respect robots.txt?

Controllable via robots.txt — OpenAI: “we recommend allowing OAI-SearchBot in your site’s robots.txt file” so your pages can appear in ChatGPT search results.

Allow or block OAI-SearchBot

# Allow (recommended if you want OAI-SearchBot traffic)
User-agent: OAI-SearchBot
Allow: /

# Block
User-agent: OAI-SearchBot
Disallow: /

Verifying it’s really OAI-SearchBot

OpenAI publishes IP ranges at openai.com/searchbot.json. User-agent strings can be spoofed by anyone — identity claims are only trustworthy when the source IP matches the operator’s published ranges.

What site owners should know

OAI-SearchBot builds the index behind ChatGPT search — allowing it is what makes your pages retrievable and citable in ChatGPT answers, independent of the GPTBot training question.

Scan your site to see how your robots.txt, content negotiation, and discovery files treat AI agents — including whether you’re blocking agents you meant to allow. Background: the robots.txt guide and agent readiness.

All AI agent user-agents

ClaudeBot · Claude-User · Claude-SearchBot · GPTBot · OAI-SearchBot · ChatGPT-User · PerplexityBot · Perplexity-User · Google-Extended · Google-Agent · Meta-ExternalAgent