AI Agent Architecture: 7 Parts of a Production Agent
AI agent architecture with diagrams: the seven parts of a production agent, five workflow patterns, single vs multi-agent, and two real systems I run.
An AI agent's architecture is a loop: a language model reads its instructions and the current state, decides on an action, calls a tool, reads the result, and repeats until the task is done or it needs a human. Around that loop sit the parts that make it safe to run in production: memory, guardrails, human approval, evaluation and observability. Get the loop right and the agent is useful. Get the parts around it right and you can trust it with real work.
I build and run agents in production: Pair runs a streaming agent with long-term memory, ServeStack Studio is a multi-agent pipeline that turns an idea into a video ad, and ServeStack exposes an MCP server so outside AI assistants can place orders. This guide is the architecture I use, with the trade-offs I've hit.
The autonomy scale: workflow, agent, multi-agent
Anthropic's widely cited Building effective agents draws the line clearly: in a workflow, code decides the path and the model fills in steps; in an agent, the model decides the path. Most real systems sit somewhere on this scale.
| Level | Architecture | Who decides the next step | Cost and risk | Label |
|---|---|---|---|---|
| 1 | Single model call | Nobody: one prompt, one answer | Lowest | Best for: classify, extract, summarise |
| 2 | Workflow (chained calls) | Your code | Low, predictable | Best for: fixed business processes |
| 3 | Single agent with tools | The model, inside limits you set | Medium | Best for: support, ops, research tasks |
| 4 | Agent with memory and approvals | The model, with a human on risky steps | Medium | Best for: assistants that act for real users |
| 5 | Multi-agent system | An orchestrator agent delegating to specialists | Highest | Best for: long jobs with distinct stages |
Start one level lower than you think you need. Every level up adds cost, latency and new ways to fail. Move up only when a test case fails because the simpler design can't handle it.
AI agent architecture diagram
┌───────────────────────────────────────────┐
user / event ──▶│ 1. MODEL (reasoning: Claude Sonnet, │
│ Opus or Haiku) │
│ 2. INSTRUCTIONS (system prompt, rules, │
│ skills) │
└───────┬───────────────────────▲───────────┘
│ decides an action │ reads result
▼ │
┌──────────────────────────┐ │
│ 6. GUARDRAILS + APPROVAL │──deny────┤
│ (limits, permissions, │ │
│ human-in-the-loop) │ │
└──────────┬───────────────┘ │
│ allowed │
▼ │
┌──────────────────────────┐ │
│ 3. TOOLS │──────────┘
│ (APIs, MCP servers, DB, │
│ search, code) │
└──────────┬───────────────┘
│
┌─────────────┴──────────────┐
▼ ▼
┌───────────────┐ ┌────────────────────────┐
│ 4. MEMORY │ │ 5. ORCHESTRATION LOOP │
│ short: context│ │ (state, retries, stop │
│ long: vector │ │ conditions, queues) │
│ DB, profiles │ └────────────────────────┘
└───────────────┘
7. EVALUATION + OBSERVABILITY wrap everything:
traces of every step, test cases run on every change
The 7 components of an AI agent
1. The model
The reasoning engine. Pick by task, not by brand loyalty: a fast, cheap model for routing and simple steps, a stronger one for planning and hard judgment. On the Claude API in November 2026, Haiku 5.5 costs $0.10 per million input tokens and Sonnet 5.5 costs $2, so the choice moves your bill by an order of magnitude (prices).
Best for: the decisions only language understanding can make. Watch out: using the biggest model everywhere.
2. Instructions
The system prompt, rules and any skills loaded on demand. This is where the agent's job, tone, limits and escalation rules live. Write it like onboarding notes for a new hire: what good looks like, what never to do, when to ask.
Best for: behaviour you can describe in words. Watch out: one giant prompt that tries to cover every case. Split know-how into skills that load only when relevant.
3. Tools
Tools are how the agent acts: call your API, query a database, search the web, run code, send a message. Each tool needs a clear name, a description that says when to use it, and a strict input schema. Expose tools through function calling inside your app, or through an MCP server when several apps and assistants should share them.
Best for: anything the agent has to read or change. Watch out: broad tools like "run any SQL". Narrow, task-sized tools are safer and the model uses them better.
4. Memory
- Short-term memory is the conversation and tool results in the model's context window. Trim and summarise it on long tasks, or the agent gets slower, pricier and more confused.
- Long-term memory persists across sessions: facts about a user, past decisions, documents. Usually a database plus vector search (I use Postgres with pgvector in Pair), written by the agent through a "remember" tool and read back by retrieval at the start of each turn.
Best for: assistants that work with the same people over time. Watch out: storing everything. Decide what is worth remembering, and let users see and delete it.
5. The orchestration loop
The code that runs the loop: send state to the model, execute the chosen tool, append the result, repeat. It also owns the stop conditions (task done, step limit, budget limit, needs a human), retries, timeouts and, for long jobs, a queue so work survives restarts. The Claude Agent SDK gives you this loop ready-made; Pair runs its own on NestJS with BullMQ and Redis.
Best for: reliability. Watch out: no step or budget limit. A confused agent will happily loop until your bill tells you.
6. Guardrails and human approval
Permissions on every tool, spending and rate limits, input and output checks, and a human approval step for anything irreversible: refunds, emails to customers, deletions, payments. On ServeStack's MCP checkout, the agent builds the order but the customer confirms payment on a real checkout page; the model never handles a card.
Best for: letting the agent act on real systems. Watch out: trusting tool output. Text from web pages, emails and documents can carry prompt injections, so one tool's output should never silently trigger another tool's write.
7. Evaluation and observability
- Evaluation: a set of real test cases (inputs plus what a good outcome looks like) that runs before launch and after every prompt, tool or model change.
- Observability: a trace of every run: each model call, tool call, input, output, latency and cost.
Best for: knowing whether a change made the agent better or worse. Watch out: skipping this. It is the part most quotes leave out, and the reason most agents quietly get worse after launch.
Agentic workflow patterns
These five patterns from Anthropic's guide cover most production designs. Combine them as needed.
| Pattern | How it works | Label |
|---|---|---|
| Prompt chaining | Step 1's output feeds step 2, with checks between | Best for: fixed multi-step tasks like draft, then translate |
| Routing | A first call classifies the input and sends it to a specialised path | Best for: support queues with distinct request types |
| Parallelization | Several calls run at once and results are combined or voted on | Best for: speed, or checking an answer several ways |
| Orchestrator-workers | A lead model splits the task and delegates subtasks to workers | Best for: tasks whose parts aren't known in advance |
| Evaluator-optimizer | One call produces, another critiques, repeat until it passes | Best for: writing, code and anything with a clear quality bar |
Single agent vs multi-agent architecture
A multi-agent system splits work across specialist agents, each with its own instructions, tools and context, coordinated by an orchestrator.
| Single agent | Multi-agent | |
|---|---|---|
| Build and debug effort | Lower | Higher: more prompts, more hand-offs |
| Token cost | Lower | Higher: several agents each read context |
| Context per agent | One crowded window | Each agent sees only what it needs |
| Good at | Focused tasks with a few tools | Long jobs with distinct specialist stages |
Rule of thumb: go multi-agent when one agent's instructions or tool list has become so large that it starts making mistakes, or when stages genuinely need different expertise.
Two architectures I run in production
Pair: single agent with long-term memory
Pair is a dating companion on iOS. One agent per user, streaming replies, with long-term memory in Postgres and pgvector so it remembers what the user shared weeks ago. Background work (memory writes, moderation) runs on queues so the chat stays fast. Level 4 on the scale: one agent, memory, and hard limits on what it may suggest.
ServeStack Studio: multi-agent pipeline
ServeStack Studio turns a restaurant's idea or a current trend into a video ad. A Python engine of cooperating agents (a director, a copywriter and an autopilot) moves through trend research, concept, script and storyboard, then generates images and video and composes the final ad. Level 5: each stage has its own instructions and tools, and the director decides what happens next. It earns its complexity because the stages really are different jobs.
How to choose your architecture
- Write down the task and ten real examples of it, with what a good result looks like. These become your first test cases.
- Try level 1 or 2 first. If a single call or a fixed workflow passes the tests, stop there.
- Add tools one at a time, each with a narrow purpose and a clear description.
- Add approval steps before anything irreversible.
- Add memory only if users return and the agent needs to know them.
- Split into multiple agents only when one agent fails the tests because it is overloaded.
- Trace everything from day one. You can't fix what you can't see.
Keep going
- Claude Agent SDK: the orchestration loop, tools and permissions, ready-made.
- MCP vs API and what an MCP server is: how agents get tools.
- How to build a RAG chatbot: retrieval, the most common kind of agent memory.
- How much an AI agent costs and Claude API pricing.
- Claude Code vs Cursor: two coding agents built on this architecture.
- Want one built? AI agent development, AI integration or tell me the workflow.
Frequently asked questions
What is AI agent architecture? It is the design of the parts an AI agent is made of and how they connect: a model, instructions, tools, memory, an orchestration loop, guardrails with human approval, and evaluation and observability. The core is a loop where the model picks an action, a tool runs it, and the model reads the result.
What is agentic AI architecture? The same idea at system level: how one or more agents, the workflows around them, their tools and the people who approve their actions fit together. It covers single agents, multi-agent systems and the workflow patterns that connect them.
What are the main components of an AI agent? Model, instructions, tools, memory, the orchestration loop, guardrails and human approval, and evaluation with observability.
What is the difference between an agentic workflow and an AI agent? In a workflow, your code decides the sequence of steps and the model fills them in. In an agent, the model decides which step comes next. Workflows are cheaper and more predictable; agents handle tasks whose steps you can't know in advance.
When should I use a multi-agent architecture? When a single agent's instructions and tools have grown so large that it starts making mistakes, or when the job has distinct stages that need different expertise. Otherwise a single agent is cheaper and easier to debug.
How do AI agents remember things? Short-term, through the conversation in the model's context window. Long-term, through a database the agent writes to and reads from, usually with vector search so it can find relevant memories by meaning.
I design and build production AI agents with the Claude API, the Claude Agent SDK and MCP. Send me the workflow you want automated and you'll get an architecture, a fixed price for the first version and an estimate of monthly running costs.
Get your agent architected and builtWritten by Antonio Kodheli
Full-stack web developer in Boston. I build modern web apps with TypeScript, Next.js, and the Claude API, and write about it here.