ChatGPT
OAI-SearchBot · ChatGPT-User
OAI-SearchBot supports search; GPTBot controls training access. Check search access, initial HTML and the source link actually used in an answer separately.
What to measure
Neutral questions with web search enabled; explicit source links in the answer.
Official documentation — ChatGPTClaude
Claude-SearchBot · Claude-User
Claude-SearchBot supports search, Claude-User handles user requests and ClaudeBot supports training. Review their robots.txt policies separately.
What to measure
A web-enabled sample with model, date and source links recorded.
Official documentation — ClaudePerplexity
PerplexityBot · Perplexity-User
PerplexityBot supports search. Perplexity-User generally ignores robots.txt for user-requested fetches. Server and firewall records are also needed to establish actual access.
What to measure
Distinguish a cited answer source from a URL only encountered during research.
Official documentation — PerplexityDeepSeek
Search API · Fetch API
DeepSeek offers its own search and page-fetch APIs. The official documentation we reviewed did not establish a webmaster submission route or verified robots.txt bot token; do not invent one.
What to measure
Record search-result retrieval separately from citations in a final answer. API access is required.
Official documentation — DeepSeekGoogle AI Overviews / AI Mode
Googlebot · Search Console
Pages must be indexed and eligible to appear with a snippet. Also check inclusion in Search generative AI features in Search Console. Standard SEO applies; no special AI file or schema is required.
What to measure
Search Console indexing and the separate Generative AI performance report. Do not label all Google traffic as AI traffic.
Official documentation — Google AI Overviews / AI ModeGemini
Google-Extended
Google-Extended controls Gemini training and certain grounding uses. It is not a Google Search ranking signal; keep this choice separate from Googlebot access.
What to measure
Source-backed answer samples; keep Gemini app sessions and API settings separate.
Official documentation — GeminiBing / Microsoft Copilot
bingbot · IndexNow
Provide accessible canonical pages for Bing indexing. IndexNow notifies participating search engines about changed URLs; it does not guarantee indexing or Copilot recommendations.
What to measure
Bing Webmaster Tools AI Performance: citations, pages and grouped grounding queries.
Official documentation — Bing / Microsoft CopilotGrok
Web Search · kaynak bağlantıları / citations
Grok provides web search and page browsing tools. We did not verify a dedicated webmaster submission route in the reviewed documentation. General accessibility and useful, verifiable content are the baseline.
What to measure
Separate all encountered sources from sources actually cited in the answer; do not merge app and API samples.
Official documentation — GrokStart with access and useful content
First check crawler policy and the initial HTML response. Then explain who each service helps, its scope, process, pricing conditions and verifiable examples. Add content when it answers a real buyer question or supplies missing information.
A user-agent name alone does not verify bot identity. Use server records and the provider’s verification method. An llms.txt file or permission for a training bot alone does not guarantee recommendations.
Open the AI crawler check · Review service scopes