Google-CloudVertexBot

Q: What is Google-CloudVertexBot?

Google-CloudVertexBot is Google's owner-commissioned crawler — it crawls a site because its owner (or a customer building on the operator’s platform) asked for it to be ingested. It obeys robots.txt and affects no search or training defaults.

Q: What is Google-CloudVertexBot's user-agent string?

Google-CloudVertexBot (vendor-documented at developers.google.com (common crawlers reference); no Mozilla wrapper is documented)

Google · Owner-commissioned crawler · Updated 2026-07-19

Google-CloudVertexBot is Google’s owner-commissioned crawler — it crawls a site because its owner (or a customer building on the operator’s platform) asked for it to be ingested. It obeys robots.txt and affects no search or training defaults.

User-agent string

Google-CloudVertexBot

Source: vendor-documented at developers.google.com (common crawlers reference); no Mozilla wrapper is documented.

Does Google-CloudVertexBot respect robots.txt?

Listed among Google’s common crawlers, which always obey robots.txt for automatic crawls. Google’s docs also list Googlebot as a matching token for it — a Googlebot disallow catches this crawler too.

Allow or block Google-CloudVertexBot

# Allow (recommended if you want Google-CloudVertexBot traffic)
User-agent: Google-CloudVertexBot
Allow: /

# Block
User-agent: Google-CloudVertexBot
Disallow: /

Verifying it’s really Google-CloudVertexBot

Google publishes common-crawler IP ranges (common-crawlers.json) and supports reverse-DNS verification (crawl-***.googlebot.com). User-agent strings can be spoofed by anyone — identity claims are only trustworthy when the source IP matches the operator’s published ranges.

Fetch the published ranges and check a suspect IP against them:

curl -s https://developers.google.com/static/crawling/ipranges/common-crawlers.json

What site owners should know

Crawls only sites a Vertex AI customer asked Google to ingest for their agent — owner-commissioned, not autonomous. It has no effect on Google Search or training defaults. Don’t confuse it with Google-Extended, the robots-only training token.

Scan your site to see how your robots.txt, content negotiation, and discovery files treat AI agents — including whether you’re blocking agents you meant to allow. Background: the robots.txt guide, how AI agents browse, and agent readiness.

All AI agent user-agents

ClaudeBot · Claude-User · Claude-SearchBot · GPTBot · OAI-SearchBot · ChatGPT-User · PerplexityBot · Perplexity-User · Google-Extended · Google-Agent · Google-CloudVertexBot · Meta-ExternalAgent · Meta-ExternalFetcher · MistralAI-User · Applebot · Applebot-Extended · Amazonbot · bingbot · DuckAssistBot · CCBot · Bytespider