A vector database stores meaning instead of rows and columns. If RAG is the librarian that walks off and fetches the right document, the vector database is the library: every call transcript and signed contract you own, filed by what it means rather than by what it says.
Vector database vs relational database
A relational database is a grid. Columns for price, rows for parts. You want the price of a condenser fan motor, you run down the column and read it off. Two dimensions. Everyone already knows how that one works.
A vector database is built to hold meaning and context. A job note, a photo of a roof, a lease renewal, a voicemail. All of it goes in. Instead of two dimensions it gives you up to four or five thousand.
| Relational database | Vector database | |
|---|---|---|
| Shape | Rows and columns | Arrays of numbers |
| Dimensions | 2 | Up to 4,000 or 5,000 |
| Retrieval | You match a value | You measure a distance |
| Good at | The price of a part | What somebody meant |
Each entry is an array, which is a long series of numbers. Read them as coordinates. There are roughly 4,000 of them and together they signify one meaning. Put a dog, a cat, a squirrel and a leaf into the same vector database and the dog, the cat and the squirrel sit close together while the leaf sits a long way off. The model does arithmetic on those distances, and that is where the meaning comes from.
How does a vector database work?
The embedding model is the piece that does the work, and it sits on both sides of the trip. It is a small model whose only job is to take tokens and return coordinates.
In goes a document, a chunk of text, an image, sometimes a sentence or two. Three steps from there:
- Embed. The embedding model reads the tokens and returns their coordinates across all 4,000 dimensions.
- Store. Those coordinates go into the vector database, next to the document they came from.
- Measure. Your question goes through the same embedding model, and the vector database hands back the data points sitting closest to it.
Retrieval is the same trip in reverse. Nothing about your question gets matched word for word. It gets placed, and whatever is standing nearby comes back.
How does semantic search find the right answer?
Take an agent that looks after your billing. Somebody asks it who cancelled last month. Sitting in the data are three labels: renewed plan, cancelled plan, voided purchase.
Renewed plan shares a word with cancelled plan and means the opposite of it. Cancelled plan and voided purchase share no words at all and mean the same thing. The vector database files the second pair close together and the first pair far apart, so the agent returns the cancellations and the voided purchases, and leaves the renewals where they are.
What is a vector database used for?
Support tickets, internal docs, product specs, past contracts: that is what we point these agents at day to day. Sometimes a ticket and a doc answer a question together, sometimes one on its own is enough.
Keep going and the vector database becomes a memory layer for the whole company. Every internal meeting, every sales call, stored and traversed by meaning. Ask what you promised the prospect on last week’s call and the vector database does not drag back the entire transcript. It goes to the end of the call, because in the way we build these systems time is one of the dimensions the vector database searches on.
That last part solved a problem we expected to have. When vector databases arrived we assumed we would spend our lives deleting stale records so the agent stopped quoting them. We don’t. Ask for the latest update on a client and you get the current one, not the write-up from three months ago, and nothing had to be deleted for that to happen.
Which vector database should you use?
Not one you build yourself. The names an engineer will put in front of you are Pinecone, pgvector, Weaviate, Qdrant, Chroma and Milvus. Pinecone is hosted, so somebody else runs it. pgvector is an extension for Postgres, so if your data already sits in Postgres you are adding a capability rather than another service. Weaviate, Qdrant, Chroma and Milvus are open source and run either on your own infrastructure or on theirs.
The mechanics in this lesson are the same in all of them. Coordinates in, distance measured, nearest entries out.
Do you need a vector database?
If you want an agent answering questions out of your own material, yes, because RAG has to have a library to walk into. If the agent only ever needs one refund policy, paste the policy into the prompt and stop. The line sits where nobody in the building can tell you which file holds the answer.
