Denshin / Blog / AI

AI agent architecture: tools, memory and the loop

The loop is the easy part of an agent. The hard parts are the tools you expose, the memory you carry between turns, and the permission layer around anything that writes. A working architecture for a production agent, including tool design, error messages as prompts, and when a state machine wins instead.

Denshin Engineering · Engineering Team · 27 August 2026 · 8 min read

Most agent tutorials show you the loop and stop there, as if the loop were the hard part. It is not. The loop is about thirty lines of code. The hard parts are the tools you expose, the state you carry between turns, and the guardrails around anything that writes. This post is a working architecture for a production agent, written from the perspective of a small team that has to run the thing afterwards.

What an AI agent's architecture actually consists of

A production agent is four things: a loop that runs until a stop condition, a set of tools the model can call, a memory strategy that decides what the model sees on each turn, and a permission layer that decides which tool calls actually execute. The model is a component inside that, not the system itself. If you are debugging agent behaviour, you are almost always debugging one of those four, not the model.

We assume here that you have already decided an agent is the right shape for the problem. If you are still weighing that, start with agentic AI explained: what actually works in production, which covers when a plain workflow wins instead.

The core loop: observe, decide, act, check

Strip away the framework and every agent is the same cycle:

Observe. Assemble the context: the task, relevant state, results of previous tool calls, remaining budget.
Decide. Call the model. It either produces a tool call or a final answer.
Act. Execute the tool call through your permission layer, capture the result and any error.
Check. Update state, decrement the budget, test the stop condition in code, then repeat.

The fourth step is the one people skip, and it is the one that separates a demo from a system. The stop condition must live in your code, not in the model's judgement. "The model said it was done" is not a stop condition, it is a claim to verify. Whatever "done" means for your task, express it as an assertion you can run: the ticket exists, the file compiles, the record matches the schema, the totals reconcile.

Everything else in the loop is bookkeeping, and it should be boring: a turn counter, a token counter, a wall-clock deadline, and a structured record of every tool call and its outcome. That record is your trace, and you will read it far more often than you expect.

Tool design is the real skill

Tools are the agent's API surface, and the same instincts that make a good HTTP API make a good tool: narrow purpose, explicit inputs, honest errors. The difference is that your consumer is a language model reading natural language descriptions, so the documentation is part of the implementation.

Narrow tools beat one god-tool

A single query_database tool that accepts arbitrary SQL looks flexible and behaves terribly. The model has to know your schema, it has to get the dialect right, the blast radius is everything, and when it fails you cannot tell whether it was a bad plan or bad SQL. Three tools called find_customer_by_email, list_invoices_for_customer and get_invoice_lines constrain the model into valid paths, are individually testable, and give you meaningful logs.

The counterweight is tool sprawl. Past roughly a dozen tools, selection accuracy starts to slip and descriptions begin to overlap. When you reach that point, either split into subagents with their own toolsets or collapse several near-identical tools into one with a well-typed enum parameter.

Typed inputs, validated at the boundary

Define every tool input as a schema, validate it before executing, and return a readable validation error rather than throwing. We use Zod for this on the TypeScript side for the same reasons we use it everywhere else, which we wrote about in why Zod replaced our hand-rolled validators: one definition drives the runtime check and the static type.

{
  "name": "refund_invoice",
  "description": "Refund a paid invoice. Refunds are not reversible. Amount is in paise. Use get_invoice first to confirm the invoice is in state 'paid'.",
  "input_schema": {
    "type": "object",
    "properties": {
      "invoice_id": { "type": "string", "description": "Invoice id from list_invoices" },
      "amount_paise": { "type": "integer", "minimum": 1 },
      "reason": { "type": "string", "maxLength": 200 }
    },
    "required": ["invoice_id", "amount_paise", "reason"]
  }
}

Note what the description is doing: it states the unit, warns that the action is irreversible, and tells the model which tool to call first. Descriptions are prompt real estate. Write them like you are onboarding a competent contractor who cannot ask you questions. The mechanics of getting reliable structured arguments out of a model are worth their own read in structured outputs and tool calling.

Error messages are prompts too

When a tool fails, the error text goes straight into the model's context and shapes the next decision. A bare "500 Internal Server Error" gives it nothing to work with, so it guesses. Compare that with "Invoice inv_123 is in state 'draft', not 'paid'. Refunds only apply to paid invoices. Call list_invoices with status=paid to find a refundable one." The second one is recoverable. Say what failed, why, and what the sensible next action is. Never return an empty result where an error occurred: an empty list reads as "there is nothing there", which is a different and much more dangerous fact.

Memory tiers, and why stuffing it all in context fails

"Context windows are huge now, just put everything in" is the most expensive shortcut in this field. Long context costs money on every single turn, it is where relevant details get lost among irrelevant ones, and it grows monotonically until the run collapses. Treat memory as tiers with different lifetimes and costs.

Tier	What lives there	Lifetime	Failure if misused
Working context	Task, recent turns, current tool results	This turn	Cost and dilution as it grows
Scratchpad or state file	Plan, findings so far, checklist of what is done	This run	Repeated work, lost progress on restart
Retrieval	Documents and records fetched on demand	Per query	Wrong or stale chunks quietly poison answers
Durable store	Facts, preferences, audit trail in your database	Across runs	Stale facts treated as current

The scratchpad tier is the one teams most often skip and most benefit from. Have the agent write its plan and its progress to an explicit structured object, and reload a summary of it each turn instead of replaying the whole transcript. It survives a crash, it is readable by a human during an incident, and it keeps the working context roughly flat as the run gets longer. Deciding what goes into the window on each turn is a discipline in itself, covered in context engineering.

One agent, subagents, or a plain state machine

Three orchestration shapes, in increasing order of how often people reach for them versus how often they should.

Single agent, many tools. The default. One loop, one context, one trace to read. Works well up to roughly a dozen tools and tasks that fit in one coherent run.
Subagents. A parent delegates a bounded piece of work to a child with its own toolset and its own context, and gets back a short result. Worth it when a subtask generates a lot of intermediate noise the parent does not need, or when two subtasks need genuinely different tools or permissions. It costs you a harder-to-follow trace and more tokens overall, so justify each one.
Plain state machine. Fixed steps, model calls inside the steps. If the sequence is known, this wins on cost, latency, testability and debuggability. A surprising number of "agent" projects are this with extra steps.

Retries, idempotency and timeouts

An agent will call the same tool twice. It will retry after a timeout that actually succeeded. It will lose the thread and redo step two. Assume all of that and design the tools accordingly.

Make writes idempotent. Accept an idempotency key derived from the task and the arguments, and return the original result on a repeat rather than performing the action again.
Separate retryable from terminal errors. A timeout or a rate limit is worth retrying with backoff, in your code, without spending a model turn. A validation error is not: hand it to the model with a clear message so it can change the plan.
Time-box every tool. One hanging call should not consume the run's entire wall-clock budget. Return a timeout error the model can reason about.
Cap retries per tool and per run. Otherwise the model will happily retry a permanently broken dependency until the budget is gone.

Permission gating for destructive actions

The permission layer sits between the model's chosen tool call and its execution, and it is code, not prompting. Classify every tool by blast radius: read-only, reversible write, irreversible or externally visible. Read-only runs freely. Reversible writes run automatically but are logged with enough detail to undo. Irreversible actions, anything touching money, anything sending a message to a customer, require an approval step or a hard rule.

Prompt instructions are not a security control. "Never delete production data" in a system prompt is a request, not a boundary, and it can be argued out of the model by content it reads from a tool result. Enforce it with scoped credentials and an allowlist. Give the agent its own identity with the narrowest permissions that let it do the job, and make sure every action it takes is attributable to that identity in your audit log.

What to do next

Write down your tool list with a blast radius label on each one, then check that the irreversible ones are gated in code.
Rewrite your three worst tool error messages to say what failed, why, and what to try next.
Add an explicit scratchpad object and stop replaying the whole transcript each turn.
Add an idempotency key to every write tool.
Put turn, token and wall-clock limits in the loop, and make hitting one a structured failure that returns partial state.

None of this is exotic. It is ordinary backend discipline applied to a caller that improvises. If you are building an agent into a product and want a review of the tool surface and the gating before it touches real data, get in touch.

Tags: AI Agents, Agent Architecture, Tool Calling, LLM, AI Engineering

All posts · Work with Denshin