The problem
Cloudflare's managed challenge (bot protection) blocks automated HTTP requests — including those from AI agents. If your site is behind Cloudflare with default settings, agents can't reach your API endpoints, payment gates, or discovery files.
agentgrade detects this per-probe. When a check is blocked, you'll see "Blocked by Cloudflare" instead of a pass/fail result. Your score may appear low not because capabilities are missing, but because Cloudflare prevented verification.
What gets blocked
Cloudflare's challenge triggers on requests that:
- Don't execute JavaScript (all API/agent requests)
- Lack browser fingerprints (TLS, headers, cookies)
- Come from cloud/datacenter IP ranges
This means every agent-facing endpoint is potentially blocked: MCP, OpenAPI, llms.txt, x402.json, payment gates, etc.
How to fix it
Option 1: Bypass challenge for API paths (recommended)
In Cloudflare's WAF rules, create a rule that skips the managed challenge for agent-facing paths:
Expression:
(http.request.uri.path matches "^/\.well-known/.*" or
http.request.uri.path matches "^/api/.*" or
http.request.uri.path eq "/mcp" or
http.request.uri.path eq "/openapi.json" or
http.request.uri.path eq "/llms.txt" or
http.request.uri.path eq "/agents.txt" or
http.request.uri.path eq "/skills.json" or
http.request.uri.path eq "/robots.txt")
Action: Skip → Managed Challenge
This keeps bot protection on your HTML pages while allowing agents to reach machine-facing endpoints.
Option 2: Allowlist specific User-Agents
Create a WAF rule that skips the challenge for known agent User-Agents:
(http.user_agent contains "agentgrade" or
http.user_agent contains "Claude" or
http.user_agent contains "GPTBot")
Option 3: Use Cloudflare's Bot Management tiers
Cloudflare Enterprise and Business plans offer more granular bot management that can distinguish between "good bots" (API clients, agents) and malicious bots.
Verifying the fix
After updating your Cloudflare rules, scan your site again on agentgrade. Previously blocked checks should now show pass/fail results instead of "Blocked by Cloudflare."