Denshin / Blog / AI
Prompt engineering best practices that survive model upgrades
Some prompt techniques keep working when the model underneath changes. Others were only ever workarounds for one version's quirks. Here is how to tell them apart, plus the operational practices that make a model upgrade routine: prompts in the repo, versioned, tested against an eval set.
Denshin Engineering · Engineering Team · 27 August 2026 · 7 min read
Every team that has shipped LLM features for more than a year has the same experience: a model upgrade lands, and half the prompt library gets slightly better while a few prompts fall off a cliff. The ones that break are almost never the plain, boring ones. They are the clever ones, tuned against the quirks of a specific model version. This post separates the prompt techniques that keep working across upgrades from the ones with a shelf life, and covers the operational practices that make an upgrade a routine event rather than a fire.
What makes a prompt technique durable
A durable prompt technique is one that gives the model information or structure it genuinely needs to do the task. A rotting technique is one that exploits a behavioural quirk of a particular model version. The first survives an upgrade because the underlying need does not change. The second breaks because the quirk was never a contract.
That is the whole test, and you can apply it to any trick you read about online. Ask: if the model were simply better at reasoning and instruction following, would this still help? If the honest answer is no, it is a workaround with an expiry date.
The techniques that keep working
Be specific about the task and the output contract
Most bad outputs are underspecified requests. Say what the input is, what the output should contain, what format it must be in, and what it must not include. If a downstream system parses the result, do not describe the shape in prose, enforce it. Schema-constrained output is far more reliable than asking politely for JSON, and it is the subject of structured outputs and tool calling.
Show examples of what good looks like
A couple of well-chosen examples communicate tone, granularity and edge-case handling faster than three paragraphs of description. Keep them few, keep them representative, and make sure they demonstrate the boundaries of the task rather than the easy centre of it. Examples that only show the obvious case teach the model nothing it did not already assume.
Give the model the material instead of relying on recall
If the answer depends on your pricing, your policy or your codebase, put that text in the request. Do not rely on the model remembering something it saw in training, and do not rely on it remembering your product at all. This is the single most reliable quality improvement available, and it is also the most upgrade-proof, because a better model is better at using material you supply. How you select that material is its own discipline, covered in context engineering.
Let it reason before answering, where the task warrants it
For multi-step problems, asking for the working before the conclusion improves accuracy. Note the qualifier. For classification, extraction and short lookups it usually adds latency and cost for nothing. Also, the mechanics here are moving: several vendors now expose reasoning as a model setting or a separate mode rather than something you prompt for, so as of writing in August 2026, check the current provider documentation before hand-rolling it into your prompt.
Decompose multi-step work
One prompt that classifies, extracts, validates and writes a summary is a prompt with four ways to fail and no way to tell which one did. Split it. Each step becomes independently testable, independently cacheable, and often independently routable to a cheaper model. The cost of an extra call is usually less than the cost of debugging a monolith.
Say what to do when it does not know
Models default to producing an answer. If you do not define the escape hatch, you will get confident guesses. Give an explicit instruction and an explicit output for the case: return an empty result, return a specific token, call a clarification tool, escalate to a human. Then test that path deliberately, because it is the one users notice most.
The techniques that rot
| Technique | Why people use it | Why it rots |
|---|
| Magic phrases ("take a deep breath", "you are a world-class expert") | Someone measured a small gain on one model | Tied to a training artefact, not a capability |
| Threats and bribes ("I will lose my job", "I will tip you") | Anecdotal reports of better effort | Vendors actively train this out, and it makes prompts embarrassing to read |
| Heavily tuned few-shot sets | Fixed a specific failure on a specific version | The examples encode the old model's blind spots |
| Elaborate persona theatre | Feels like it should help | Burns tokens, and long backstories dilute the actual instructions |
| Formatting superstition (all caps, repeated instructions) | Worked once under a deadline | Contradicts itself as prompts accumulate edits |
A short, specific persona is fine: "You are a support agent for a booking platform. You answer only from the provided policy text." That is a scope constraint, not theatre. The rot sets in when the persona grows a name, a biography and a personality that nobody tests.
Treat prompts as code, not as content
The operational half of this is more important than the wording half, and it is where most teams are weakest.
- Keep prompts in the repository, not in a database row. A prompt is logic. It should be diffed, reviewed and released with the code that depends on it. Prompts edited live in an admin panel change behaviour with no review, no history and no rollback, and you will not be able to explain a regression a week later.
- Version them explicitly. Give each prompt an identifier and a version, and log which version produced each output. When quality moves, you want to know whether the prompt changed, the model changed, or the inputs changed.
- Pin the model version where the provider allows it. Floating aliases are convenient and will surprise you. Pin in production, upgrade deliberately, and hedge for the fact that pinned versions are eventually retired, so "pin and forget" is not a strategy either.
- Review prompt diffs like code diffs. The person who added "always be concise" to fix one complaint should have to explain it to someone who knows about the three prompts that need long answers.
The upgrade routine
An eval set is what turns a model upgrade from a gamble into a chore, which is what you want it to be. It does not have to be big. Thirty to fifty real cases with known-good outcomes, including the failures that embarrassed you in production, will catch most regressions. Build it the way how to evaluate AI agents describes, and keep it in the repo next to the prompts.
Then the routine on every model change is fixed:
- Run the eval set on the new model with the existing prompts, unchanged. Record the diff.
- Look at what regressed and ask whether the prompt was compensating for something the old model got wrong. Usually it was. Delete the compensation rather than adding a new one.
- Re-check cost and latency, not just quality. Newer is not automatically cheaper or faster, and pricing tiers move.
- Ship behind a flag with the old model one config change away.
If you are choosing between models rather than upgrading within a family, the same eval set is the input to that decision too, as argued in choosing an LLM: a decision framework, not a leaderboard.
Prompt caching, and why prompt order matters commercially
Most major providers now offer some form of caching for repeated prompt prefixes, which can meaningfully cut the cost and latency of a long, stable system prompt. The details, the discounts and the minimum sizes differ by vendor and change, so as of writing in August 2026, treat the provider docs as the source of truth rather than any number you read in a blog post, including this one.
The design consequence is stable regardless of vendor: put the parts of your prompt that never change at the front, and the parts that vary per request at the back. Interleaving a user's name into the middle of an otherwise fixed system prompt can defeat prefix caching entirely. That is a structural habit worth adopting even before you turn caching on.
What to do next
Pick your most-used prompt and run three checks this week.
- Delete everything in it that fails the durability test. If the model being smarter would make the line pointless, the line is a workaround.
- Move it into the repo with a version identifier, and log that identifier with every output.
- Write ten evaluation cases from real traffic, including two you know it currently gets wrong.
You will end up with shorter prompts, which is almost always the direction of improvement. If you have a prompt library nobody is confident enough to touch, or an upgrade you have been putting off, talk to us.
Tags: Prompt Engineering, LLM, AI Engineering, Evals, Prompt Caching
All posts · Work with Denshin