Denshin / Blog / AI
Context engineering: the skill that replaced prompt engineering
Prompt wording is no longer the hard part. The hard part is deciding what enters the context window, in what order, and at what cost. A practical guide to the context budget, the failure modes of a stuffed window, and the techniques that reclaim it.
Denshin Engineering · Engineering Team · 27 August 2026 · 7 min read
The most useful skill in shipping AI features stopped being the wording of your prompt some time ago. It is now the decision about what information enters the model's context window, in what order, and at what cost. Getting that right is the difference between a feature that answers correctly on the third turn and one that quietly degrades the moment a conversation gets long. This post is about treating the context window as an engineering constraint you budget and measure, rather than a bucket you keep pouring into.
What context engineering actually means
Context engineering is the practice of deciding what goes into a model's context window on every call: the system instructions, the tool definitions, the retrieved documents, the conversation history, and the room left for the answer. Prompt engineering asks how to phrase a request. Context engineering asks what the model should be looking at when it reads that request, and what it should not be looking at.
The shift matters because modern models are already good at following clear instructions. They are much less good at ignoring irrelevant material you handed them. Once you accept that, most of your effort moves from rewriting sentences to curating inputs.
The context window is a budget, not a bucket
Every call has a fixed number of tokens available, and every token you spend on one thing is a token you cannot spend on another. Long-context models have made the budget bigger, not infinite, and a bigger budget does not mean you should fill it. Think of a single call as an allocation across five competing lines.
| Budget line | What it buys | Where it goes wrong |
|---|
| System instructions | Role, output contract, hard rules | Grows by accretion, nobody deletes anything |
| Tool definitions | What the model can do | Twenty tools loaded when three are relevant |
| Retrieved material | Facts the model cannot recall | Whole documents pasted instead of the passage |
| Conversation history | Continuity across turns | Carried verbatim forever, including dead ends |
| Output room | Space for the answer to land | Squeezed last, so long answers get truncated |
Write those five lines down for your own feature and put a rough token number next to each. The exercise takes ten minutes and usually reveals that one line is eating half the window for very little benefit.
How a stuffed context window fails
The failures are not loud. You rarely get an error. You get answers that are slightly worse in ways that are hard to attribute, which is exactly why teams keep adding material instead of removing it.
Distraction
Irrelevant text in the window competes for the model's attention with the text that matters. If you paste a whole policy document to answer a question about refunds, the sections on shipping and warranties are not neutral. They are plausible-looking material that the model may weave into an answer.
Contradictory instructions
Long-lived system prompts accumulate rules from different incidents. Someone adds "always ask a clarifying question first" after a support escalation. Someone else adds "never ask more than one question" after a different complaint. Both survive. The model now behaves inconsistently, and it looks like model flakiness rather than a specification bug you wrote yourself.
Position effects
It is well established across the industry that models attend unevenly to very long inputs, with material in the middle of a large context often used less reliably than material near the beginning or the end. Treat that as a design constraint rather than a benchmark number: if a fact is load-bearing, do not bury it in the middle of forty pages of retrieved text.
Cost and latency creep
Input tokens are billed on every single call, and a chat feature re-sends its history each turn. A prompt that grew by a few thousand tokens is not a one-off cost, it is a multiplier on every request you will ever serve. We covered the commercial side of this in adding AI to your product without setting money on fire, and the arithmetic there applies directly here.
Techniques that reclaim context
Retrieve, do not dump
The default instinct is to give the model everything relevant. The better instinct is to give it the smallest set of passages that could support a correct answer, and to make the retrieval itself good enough that you can afford to be stingy. That means chunking and ranking deserve real attention, which is the subject of RAG chunking strategies that actually improve answers. A retrieval step that returns three precise passages beats one that returns twenty adequate ones, on quality and on cost.
Compact the history instead of carrying it
Conversation history is the line item that grows without anyone deciding it should. Two practical patterns work well. First, keep the last few turns verbatim and replace older turns with a running summary that preserves decisions and constraints rather than dialogue. Second, drop tool call payloads once they have been acted on: the model needs to know that a search happened and what it concluded, not the full JSON that came back.
Externalise state
If a piece of information needs to survive, put it somewhere durable and read it back on demand. A file, a row in a table, a scratchpad document. This is the single biggest lever for long-running agents: the context window becomes working memory rather than permanent storage, and the agent can run for hours without the window becoming a landfill. The trade-off is that you now need retrieval logic for your own state, which is a fair price.
Sub-agents for context isolation
When a task involves reading a lot to produce a little, delegate it. A sub-agent can read thirty files and return a five-line conclusion, and the parent's context only ever sees those five lines. This is why the pattern shows up in almost every serious agent design, discussed further in AI agent architecture: tools, memory and the loop. The cost is coordination overhead and a real risk of the sub-agent summarising away something the parent needed, so use it for search and analysis, not for decisions.
Load tools conditionally
Tool definitions are context too. If your agent has a large tool surface, gate it: expose the tools relevant to the current phase, or route to a smaller tool set based on a cheap classification of the request. Fewer, well-described tools produce better tool choice than a long undifferentiated list.
Put the durable rules in a file the agent always reads
For coding agents this is the highest-leverage thing you can do, and it takes an afternoon. Keep a conventions file at the root of the repository that the agent reads at the start of every session: the stack, the commands to build and test, the directory conventions, the things that are not allowed. It replaces the per-prompt re-explaining that otherwise burns tokens and gets forgotten.
# CLAUDE.md (or your agent's equivalent)
## Commands
yarn dev # all workspaces
yarn typecheck # run before proposing a diff
## Conventions
- Feature folders under src/features/<name>
- Data fetching goes through TanStack Query hooks, never raw axios in components
- Never edit generated files under dist/
Two rules keep it useful. Keep it short, because it is paid for on every session. And treat it as code: it lives in version control, it gets reviewed, and when a rule stops being true you delete it rather than adding a caveat.
How to tell whether your context is working
Context problems hide from manual testing because you test with short conversations and clean inputs. Three checks catch most of it.
- Log the assembled context, not just the prompt template. You want to see the actual bytes sent on a failing request. Most surprises are visible immediately once you look.
- Run an ablation. Take a chunk of the context out and re-run your eval set. If quality does not move, that chunk was costing you money for nothing. This is the fastest way to shrink a bloated system prompt.
- Test at length. Evaluate on long conversations and large retrieved sets, not just the happy short path. If you do not have an eval set yet, how to evaluate AI agents is the place to start, because every technique in this post is a trade-off you cannot judge without measurement.
What to do next
Pick your worst-behaving AI feature and do these four things this week.
- Print one real assembled context and read it end to end. Delete anything you cannot justify.
- Write down the five budget lines with token counts, and decide which one you are over-spending on.
- Add history compaction if the feature is conversational, or move state to a file or table if it is an agent.
- Move the durable rules out of the prompt and into a versioned file the agent reads first.
None of this requires a new model or a new vendor. It requires deciding, deliberately, what the model gets to see. If you are building an AI feature and the quality is drifting in ways you cannot explain, talk to us. It is usually the context, not the model.
Tags: Context Engineering, AI Agents, LLM, Prompt Engineering, RAG
All posts · Work with Denshin