What an AI agent actually is (and when you don't need one)

“Agent” has become the word everybody uses and few people define. Vendors call almost anything an agent now: a chatbot, a scheduled script, a form with a language model behind it. That makes it hard for business owners to know what they are buying, or whether they need it at all.

Here is the definition I keep coming back to, and it fits in one sentence:

An agent is a system where the AI decides what to do next, not just what to say next.

A chatbot answers. An agent chooses the next step, uses a tool, looks at the result, and decides again. That loop is the whole difference.

The ladder

It helps to think of AI in a business as a ladder with four rungs. Most problems belong lower than people expect.

1. A good prompt

A person asks, the model answers, the person decides what to do with it. Writing a first draft of an email, translating a product description, summarising a document. No integration, no risk, instant value.

2. A fixed workflow with AI inside

The steps are decided in advance by a human. AI handles one or two of them. For example: every new inquiry arrives, AI classifies it and drafts a reply, the draft lands in a shared inbox for approval. The path never changes; only the content does. This is where most real business value lives today.

3. A workflow with decisions

Same as above, but the AI picks between a few predefined branches. Is this a complaint, an order, or a partnership request? Each answer leads to a different, still human-designed, path.

4. An agent

The AI receives a goal and a set of tools, and decides the sequence itself. Research these ten companies, find the right contact, check if they already sell a competing brand, summarise. The path is different every time.

When you actually need rung four

Agents earn their complexity when the task is open-ended and the steps genuinely can’t be known in advance: research, investigation, multi-step troubleshooting, work that a smart junior colleague would do by poking around.

They are a poor fit when the steps are already known. If you can draw the process as a flowchart, build the flowchart. It will be cheaper, faster, easier to debug and far more predictable.

Three questions before you build one

1. Can I describe the steps? If yes, start with a workflow.
2. What happens when it’s wrong? Agents make more varied mistakes than workflows. Make sure a human checks the output wherever a mistake costs money or reputation.
3. Who reviews its work? An agent without a reviewer is not automation, it’s a liability with a nice interface.

The honest answer for most small teams in 2026: you need rung two far more often than rung four. The agent can come later, once you know exactly which judgement you want to hand over.