Crawlers

Google-Extended

Google-Extended is Google's crawler specifically dedicated to AI model training (Gemini). Distinct from classic Googlebot, it can be blocked separately via robots.txt.

What is Google-Extended?

Google-Extended is a Google robots.txt control token. It is not a robot that visits your site with its own user-agent: crawling is done by Googlebot, and the “User-agent: Google-Extended” line in your robots.txt only signals whether Google may use your content to train its AI models.

What the token lets you choose

  • Allow: Google may use crawled content to train and ground its AI models (such as Gemini).
  • Disallow: AI-training use is excluded, with no effect on classic Google Search indexing.

Blocking Google-Extended does not remove your pages from classic search results. Do not confuse it with the HTTP robots actually detected on your site, such as GPTBot or ClaudeBot.

robots.txt configuration

User-agent: Google-Extended
Allow: /

or Disallow: / to refuse AI use of your content.

Google-Extended and AI visibility

Visibility in Gemini answers and AI Overviews mostly depends on how Googlebot finds, understands and cites your pages. Model training is a separate question, handled by this token. For a complete strategy, see also the detectable AI robots (GPTBot, ClaudeBot, PerplexityBot) and their real-time tracking.

Sources: token status verified in the official Google documentation: Google-Extended; page reviewed on 12 November 2026.

How do your brands appear in AI answers?

Agencies, consultants and in-house teams: compare the answers generated for your own scenarios, languages and selected models. Our infrastructure, your scenarios, your expertise.

Go further

Discover our in-depth article on this topic

Read article