Denshin / Blog / AI
Vector databases and embeddings explained without the hype
What an embedding really is, what similarity does and does not capture, and how to choose between an in-memory index, pgvector in the Postgres you already run, a managed search service and a dedicated vector database. Plus ANN recall trade-offs, re-embedding migrations, dimension cost, metadata filtering and index sync.
Denshin Engineering · Engineering Team · 27 August 2026 · 7 min read
Vector databases arrived with a lot of noise, and the noise obscured a simple engineering question: where should you store some numbers so you can find the nearest ones quickly? That is genuinely all this is. The answer for a small product is often "in the Postgres you already run", and knowing when that stops being true is worth more than knowing any vendor's feature list. Here is the plain version, with the operational costs that demos leave out.
What an embedding actually is
An embedding is a list of numbers, typically a few hundred to a few thousand of them, produced by a model that has been trained so that texts with similar meanings get similar lists. Nothing more mystical than that. The individual numbers mean nothing you can name, and there is no dimension you can read as "formality" or "topic". What is meaningful is only the relationship between two embeddings, usually measured as cosine similarity, which is a number describing how closely two vectors point in the same direction.
Because meaning is encoded as position, "cancel my subscription" and "how do I stop being billed" land near each other even though they share almost no words. That is the whole value proposition. It is also why the technique fails in ways keyword search never would.
What similarity captures, and what it misses
Embedding similarity captures topical and semantic relatedness well. It reliably misses several things that matter in production, and every one of these has burned a real system:
- Negation. "The refund is available" and "the refund is not available" are topically near identical and will embed close together. Similarity does not encode truth.
- Exact identifiers. Error code E-4021, SKU numbers, version strings and surnames are exactly what users search for and exactly what similarity is worst at. Keyword search handles these trivially, which is the argument for hybrid retrieval.
- Recency and authority. A superseded 2023 policy and its 2026 replacement are near neighbours. The embedding has no opinion about which one is current. That is a metadata and ranking job.
- Relatedness is not relevance. A document about a topic is not the same as a document that answers the question. This is the gap that reranking exists to close.
Understanding these limits is what stops you from concluding that "the model is bad" when the retrieval layer is simply doing what it always did. We go deeper on fixing that layer in chunking strategies that actually improve answers.
Do you actually need a dedicated vector database?
Start from your corpus size and your existing stack, not from a comparison article. There are four reasonable answers and only one of them involves adopting a new piece of infrastructure.
An in-memory index
For a few thousand documents, load the vectors into memory and compute similarity by brute force. Comparing a query against a few thousand vectors is trivial arithmetic and is over before the network call to the model returns. No index, no server, no sync problem, no approximation. For internal tools and single tenant products, plenty of teams never need to leave this tier, and the ones who do usually discover the limit as an obvious slowdown rather than a mystery.
pgvector in the Postgres you already run
This is the default we recommend for most client products. Vectors become a column, similarity search becomes a query, and metadata filtering becomes a WHERE clause on the same row. You get transactions, backups, access control, migrations and one operational surface to monitor, rather than two systems that can silently disagree. The upper bound is high enough that most business applications never reach it, and the argument here is the same one we make about the boring stack winning most client work.
A managed search service
If you already run a search cluster, or you need strong keyword search and vector search together with mature relevance tuning, mainstream search engines now support vector fields directly. Reaching for what you already operate beats adding a component, and hybrid retrieval comes essentially for free.
A dedicated vector database
Justified when scale or workload genuinely demands it: many millions of vectors, heavy write throughput with live index updates, multi tenant isolation at scale, or the specific filtering and hybrid features a product offers. These are real systems solving real problems. They are simply not the first thing a team with fifty thousand chunks should install.
| Option | Sensible scale | Main benefit | Main cost |
|---|
| In-memory index | Thousands | No infrastructure at all | Rebuild on restart, single process |
| pgvector | Thousands to low millions | One database, joins and filters | Shares resources with your OLTP load |
| Managed search service | Millions | Hybrid search built in | Cluster to operate and pay for |
| Dedicated vector database | Millions and up | Purpose built performance | Another system to run and sync |
ANN indexes and the recall versus latency trade
Once brute force gets slow, you switch to approximate nearest neighbour search. The word that matters is approximate. An ANN index does not guarantee it will find the true closest vectors; it finds very close ones, very fast, and you tune how hard it looks.
Conceptually there are two dominant families. Graph based indexes such as HNSW link each vector to its neighbours and walk the graph towards the query, which gives excellent recall and speed at the cost of memory. Partition based indexes such as IVF cluster the vectors and search only the nearest few clusters, which is lighter on memory but can miss a result sitting just across a cluster boundary.
Every ANN index exposes a knob that trades recall for latency: how many neighbours to explore, how many partitions to probe. Turn it up and you find more of the true results and take longer. The mistake to avoid is treating the default as correct without measuring what recall you are actually getting. Measure it against a brute force ground truth on a sample of your own queries, because published benchmarks are run on datasets that are not yours.
The operational realities nobody demos
Re-embedding is a migration
Embeddings from different models are not comparable. Not "less accurate together", not comparable at all, because they are coordinates in different spaces. So the day you change embedding model, you re-embed the entire corpus. Plan it like a database migration: a job that can run over a large corpus without downtime, a way to write into a second index, and a cutover. Teams that did not plan for this find themselves stuck on an old model because the migration is too scary to attempt.
Dimensions cost real money
Storage scales linearly with dimension count, and for graph indexes so does memory, which is the expensive resource. A higher dimensional model is not automatically better for your task and can be several times the cost to serve. Some current models support shortening the vector at some quality cost, which is worth evaluating on your own data. Test the smaller option before assuming you need the larger one.
Metadata filtering is where architectures break
Real queries are rarely pure similarity. They are "similar to this, and visible to this user, and from the last twelve months". How a system combines filtering with ANN search matters enormously: filter first and the index may be bypassed into a slow scan, filter after and you can retrieve fifty results and have three survive the filter. This is the single strongest practical argument for keeping vectors next to your relational data, where the filter is just part of the query plan. It also matters for correctness, because permission filtering has to happen in retrieval, a point we make in RAG vs fine-tuning.
Keeping the index in sync with the source of truth
The vector index is a derived artefact. The source of truth is your database or document store. When a document is edited or deleted, the corresponding chunks must be updated or removed, and the failure mode when they are not is ugly: an AI assistant quoting a deleted policy back to a customer. Decide up front whether you write synchronously on change, queue a background job, or reindex on a schedule, and build a reconciliation job that detects drift rather than hoping it will not occur. This is ordinary derived-data discipline, the same thinking behind our notes on single-table design at small scale.
On vendor specifics, and why we are hedging
Every concrete number in this space moves. Model dimensions, per million token embedding prices, index types supported, managed tier limits and which database ships which feature all changed in the last year and will change again. As of writing in August 2026, the sensible move is to check current vendor documentation before committing, and to design so that the embedding model and the storage layer are both replaceable. Write a thin interface over "embed this text" and "find me the nearest k with these filters". That indirection costs an afternoon and buys you the ability to change your mind.
What to do next
- Count your chunks. Under roughly ten thousand, start in memory or in pgvector and stop reading vendor comparisons.
- Put an interface in front of embedding and search so both are swappable.
- Add keyword search alongside vector search early, because identifiers and codes will otherwise embarrass you.
- Measure recall against brute force on your own queries before tuning any index parameter.
- Write the re-embedding migration and the sync reconciliation job before you have a million vectors, not after.
If you are choosing where to put your vectors for a product that has to run cheaply on AWS and be maintained by a small team, get in touch. The answer is often less infrastructure than you were expecting.
Tags: Vector Databases, Embeddings, RAG, Pgvector, Semantic Search
All posts · Work with Denshin