## The definition

An **AI agent** is software that uses a large language model to plan and execute actions on a user's behalf — running searches, calling APIs, comparing prices, filling orders, making payments. The defining trait is not intelligence but *delegation*: a human states a goal, and the agent decides which sites to visit, which tools to call, and when the job is done.

That makes an agent a fundamentally different visitor from anything the web was built for. It does not browse — it *uses*. It has no eyes for your design, no patience for your clickflow, and a hard timeout where a human would wait.

## Agent vs. bot vs. crawler

The three words get used interchangeably; they describe different machines:

| | Crawler | Bot | Agent |
|---|---|---|---|
| Drives itself by | A schedule and a URL frontier | A fixed script | A goal and a plan |
| Works for | An index or dataset | Whoever wrote the script | One user, right now |
| Visits your site | Broadly and repeatedly | Wherever the script says | Only if you're useful to the task |
| Example | GPTBot, ClaudeBot | A price-monitoring script | ChatGPT booking a table |

The confusion is understandable because the same vendors run all three. OpenAI operates a training crawler (GPTBot), a search-index crawler (OAI-SearchBot), *and* the live fetcher its agents use (ChatGPT-User) — three tiers with separate robots.txt tokens and separate behavior. [The robots.txt guide](/kb/robots-txt) explains the tier model; [the user-agent reference](/agents) documents every bot in it.

## The anatomy of an agent

Four parts, each of which meets your website differently:

- **The planner** — the LLM itself. It never touches your server; it only sees what the other parts bring back.
- **The tools** — how the agent acts. If your site exposes an [MCP server](/kb/mcp) or an [OpenAPI spec](/kb/openapi), the agent can call your capabilities directly instead of scraping pages.
- **The fetcher** — the HTTP client that actually hits your server. This is where [content negotiation](/kb/content-negotiation) happens: a fetcher that asks for `text/markdown` and gets clean markdown ingests your content in one pass.
- **The wallet** — how the agent pays. A machine-readable [402 payment challenge](/kb/comparing-payment-protocols) is the difference between an agent buying from you and an agent recommending your competitor.

## What an agent needs from your website

An agent completes a task on your site exactly as far as your machine-readable surface lets it. The layers, in the order an agent encounters them: discovery files ([llms.txt](/kb/llms-txt), agent-aware [robots.txt](/kb/robots-txt)) so it can orient; callable capabilities ([MCP](/kb/mcp), [OpenAPI](/kb/openapi)) so it can act; [content negotiation](/kb/content-negotiation) so it can read; and a [payment path](/kb/comparing-payment-protocols) so it can transact. That stack is what [agent readiness](/agent-readiness) measures.

## What agent traffic looks like in your logs

Less obvious than you'd expect. Some of it announces itself — the named user-agents documented in [the reference](/agents), verifiable against published IP ranges. But a large share arrives anonymously: coding agents and desktop assistants fetch from the *user's own machine*, so the request comes from a residential IP with a generic client string like `curl` or a plain browser UA. If your logs show growing traffic from generic clients that fetch one page and never load assets, that is often agent traffic you cannot attribute. Blocking it blocks the humans who sent it.

## FAQ

### Is ChatGPT an AI agent?

ChatGPT is an assistant that *contains* agents. When it only converses, it isn't acting as one; when it searches the web, fetches your page, or completes a checkout on a user's behalf, that is agent behavior — and it arrives at your server as ChatGPT-User or an agent-mode fetcher.

### Is an AI agent the same as a chatbot?

No. A chatbot converses; an agent acts. The test is whether the software takes real-world actions (fetching, calling APIs, transacting) in pursuit of a goal, not whether it talks.

### Do AI agents render JavaScript?

Mostly no. Most fetchers read the raw HTTP response and never execute scripts, so content that only exists after JavaScript runs is invisible to them. See [JavaScript rendering](/kb/javascript-rendering).

### How do I know whether agents visit my site?

Check your access logs against the named tokens in [the user-agent reference](/agents), and watch for the anonymous pattern: generic client strings, residential IPs, single-page fetches with no asset loads. Then [scan your site](/) to see what those visitors can actually do once they arrive.

## Related

- [How AI agents actually browse the web](/kb/how-ai-agents-browse)
- [Agent readiness: the definitive guide](/agent-readiness)
- [AI agent user-agents: the reference list](/agents)