Denshin / Blog / Engineering

Adding AI to your product without setting money on fire

LLM prices fell ~80% in a year, but careless integration still burns runway. Model routing, prompt caching, evals, and the boring scaffolding that turns an AI demo into a production feature.

Denshin Engineering · Engineering Team · 25 June 2026 · 3 min read

The fastest way to turn an exciting AI feature into a frightening invoice is to wire your app straight to the most expensive model for every request and hope for the best. We add AI to client products regularly, and the difference between a feature that delights users and one that quietly burns the runway is almost entirely in the engineering around the model, not the model itself.

Here's how we think about shipping AI features that are genuinely useful without setting money on fire.

First, the good news on price

The economics have shifted hard in your favour. LLM API prices fell roughly 80% between early 2025 and early 2026. Capable mid-tier models now cost a fraction of what frontier models cost a year ago, and there's a healthy spread of options at every price point. The thing that was prohibitively expensive to prototype in 2024 is, for most use cases, cheap to prototype today.

That doesn't mean cost stops mattering. It means the waste moves from "the API is expensive" to "we're using it carelessly." The cheapest API call is still the one you never make.

Route requests to the right-sized model

The single highest-leverage cost decision is to stop sending every request to your most powerful model. Most real workloads have a long tail of easy requests and a small head of genuinely hard ones. A typical split routes around 70% of queries to a budget model, 20% to a mid-tier, and 10% to a premium model, and that alone tends to cut average per-query cost by 60-80% versus routing everything through the top tier.

The trick is classifying difficulty cheaply and falling back gracefully when the small model isn't confident. That logic is unglamorous and it's exactly where the savings live.

Cache aggressively

Every major provider now offers prompt caching: your long, stable system prompt and shared context get stored server-side and billed at a fraction of the normal input rate on repeat calls. If your app sends the same instructions on every request (and almost all of them do), caching is close to free money. Beyond that, cache whole responses for identical or near-identical inputs. A surprising amount of "AI traffic" is the same handful of questions asked over and over.

The part that actually makes it production-ready

Cost control gets the headlines, but the thing that separates a demo from a feature you can put in front of paying customers is the boring scaffolding around the model:

Evaluations. A small, honest test set of real inputs with known-good outputs, so you can change a prompt or swap a model and actually know whether you made it better or worse, instead of guessing from vibes.
Guardrails. Validation on what goes in and what comes out. The model is a probabilistic component in a deterministic system; treat its output like untrusted input, because it is.
Observability. Log tokens, latency, and cost per feature from day one. You can't optimise a number you aren't watching, and AI cost is the one that surprises teams.
Graceful failure. Models time out, rate-limit, and occasionally return nonsense. The feature has to degrade politely, not throw a stack trace at your user.

Treat the model as one component in a normal, well-engineered system, not as the system. The teams who get this right ship calm, reliable AI features. The teams who treat the model as magic ship a demo that falls over in week two.

Pick the model for the job, then leave room to switch

Default to a genuinely capable model for anything user-facing and reasoning-heavy: quality is what users remember, and on today's prices the premium is small. Use the cheap, fast models for the easy tail. Most importantly, build behind a thin abstraction so you can change models without rewriting your application; the price-performance leaderboard moves every few months, and you want to ride that, not be married to one provider.

We use AI heavily in our own delivery too (we wrote about how AI pair-programming reshaped how we ship), which is part of why we have strong opinions about doing it carefully.

If you're trying to add an AI feature that's actually useful and won't blow up your costs, tell us what you have in mind and we'll help you scope it sensibly.

Tags: AI, LLM, Product, Cost Optimization, Machine Learning

All posts · Work with Denshin