How Vector Databases Work
AdvancedA vector database stores embeddings and finds the most similar ones to a query vector, fast, even across billions of items. A brute-force search comparing the query to every stored vector is accurate but far too slow at scale. Vector databases use Approximate Nearest Neighbor (ANN) indexes — like HNSW and IVF — that trade a tiny bit of accuracy for enormous speed. They also combine similarity search with metadata filtering, so you can find "similar items that also match these constraints." They are the storage layer behind RAG and semantic search.
Think of a librarian who groups books by topic first
To find books similar to one in your hand, checking every book in the library (brute force) is impossibly slow. A smart librarian instead organises shelves into topic neighbourhoods and keeps shortcuts between related sections. To find similar books, you jump to the right neighbourhood and search only nearby shelves. You might occasionally miss one perfect match, but you find excellent results almost instantly. That neighbourhood-and-shortcut structure is what an ANN index does for vectors.
Step by Step
Key Concepts
Approximate Nearest Neighbor (ANN)
Algorithms that find very-close (not guaranteed exact) matches far faster than brute force. They trade a small, tunable amount of recall for a huge speed gain — the core of scalable vector search.
HNSW
Hierarchical Navigable Small World — a graph-based ANN index that connects vectors into navigable layers, giving fast, high-recall search. A default choice in many vector databases.
IVF (Inverted File)
An ANN approach that clusters vectors into cells and, at query time, searches only the cells nearest the query. Often combined with compression (PQ) to save memory.
Hybrid Filtering
Combining vector similarity with structured metadata filters (tenant, date, category) so results are both semantically relevant and satisfy business constraints — essential for real applications.
Key Facts
- Vector databases use approximate search because exact nearest-neighbor over millions of high-dimensional vectors is too slow — a tiny recall trade-off buys orders-of-magnitude speed.
- HNSW and IVF are the two most common index families; the choice trades memory, build time, speed, and recall.
- Metadata filtering is as important as similarity — production RAG almost always needs "similar AND matching these constraints," such as per-tenant isolation.
Real-World Applications
RAG retrieval at scale
A RAG system stores millions of document-chunk embeddings in a vector database and retrieves the top-k most relevant chunks per question in milliseconds, filtered to the right knowledge base or tenant.
Recommendations and search
An e-commerce site embeds products and uses a vector database to power "similar items" and semantic product search, combining vector similarity with filters like price range and availability.
Frequently Asked Questions
What is a vector database?
A vector database is a system built to store embeddings (numeric vectors representing meaning) and efficiently find the most similar vectors to a query. Unlike a relational database optimised for exact matches and joins, a vector database is optimised for similarity search over high-dimensional vectors, using specialised indexes and usually combining that with metadata filtering. It is the storage and retrieval layer behind semantic search, RAG, and recommendation systems.
What is Approximate Nearest Neighbor (ANN) search?
ANN search finds vectors that are very close to a query vector without guaranteeing the exact nearest ones, in exchange for dramatically faster performance. Exact nearest-neighbor search must compare the query to every stored vector, which is too slow at scale. ANN algorithms like HNSW and IVF organise the vectors so the search only examines a small, promising subset, trading a small, tunable amount of recall for orders-of-magnitude speed — which is what makes similarity search practical over millions or billions of items.
What is the HNSW index?
HNSW (Hierarchical Navigable Small World) is a graph-based approximate nearest neighbor index. It connects vectors into a multi-layer navigable graph where searches start at a coarse top layer and descend to finer layers, quickly hopping toward the query neighbourhood. HNSW offers fast search with high recall and is a popular default in many vector databases, though it uses more memory than some alternatives.
Why is metadata filtering important in a vector database?
Because real queries rarely want pure similarity — they want similar items that also satisfy constraints, such as documents belonging to a specific tenant, within a date range, or of a certain category. Vector databases combine the similarity search with structured metadata filters so results are both semantically relevant and valid for the use case. This is essential for multi-tenant RAG (isolating each customer data) and for search that must respect business rules.