Agent

People hear “agent” everywhere, but exactly what it means can be confusing. Once you understand agents, you can better judge which tasks are suitable to be automated by agents. Spoiler alert: not everything should be converted to an agent!

Key concepts

In short, Agent = LLM + tool use + autonomy. We talked about LLM models and tool use. The last missing piece is “autonomy”. Given a task, an agent has the freedom to decide how to approach the problem: which tools to use next, what step to take next, when to stop etc.

To understand autonomy better, it's helpful to contrast with “Workflows” (sometimes this is called “LLM chain”). Workflows = LLM + tool use, but no autonomy. In a workflow, the LLM follows predefined steps designed by the human to perform a task.

Lets use a kitchen analogy. In a workflow, Bob-the-cook is following a recipe that a human wrote. The recipe says: first do this, then do that, then do the next thing. Bob may use tools, but the sequence is already designed for him.

In an agent, Bob is acting more like a chef. You tell him the goal: “Make dinner for six people with these dietary constraints.” You do not specify every step. Bob has to inspect the kitchen, decide what to cook, choose the order of actions, adapt if ingredients are missing, and ask for help only when needed.

Neither are strictly better than the other.

Intuitively, workflows are more repeatable/stable. If you e.g. need the same weekly report created the same way every time, use a workflow. You want repeatability. If the task cannot be cleanly broken down into fixed, deterministic steps, an agent might be better. E.g. building a customer-facing chat bot. Since we don’t know ahead of time what the customer is going to ask for (e.g. product query, purchase, refund), agent is likely a better choice: we let the agent decide what to do.

Advantage of agent: more flexible. Downside: “with great power comes great responsibility”. Agents can mess up, so if the task has high stakes, might want to consider workflow (or add human-in-the-loop, see Guardrails).

Note: sometime people do not make this difference between workflow/agent too clearly. E.g. refer to sth as an agent when its actually a workflow. This is normal since there is no single standard definition of agent.

Note 2: some people may argue Agents are strictly better than Workflows. This is subjective and depends on the situation. (But if your interviewer/boss believe this, you might not want to say they are wrong :p)

(Triple quiz)