How Vector Search Works
IntermediateVector search finds items whose meaning is closest to a query by comparing embedding vectors rather than matching keywords. You embed the query into the same vector space as your content, then find the nearest vectors — the semantically most similar items. At scale this uses approximate nearest neighbor algorithms for speed. The best systems combine vector search with keyword search (hybrid search) and add a re-ranking step, getting both the recall of semantic matching and the precision of exact terms.
Think of finding the nearest cafes on a map
To find cafes near you, you do not read a list of every cafe address — you look at a map and pick the closest points. Vector search is the same, but the "map" is a space of meaning and each item is a point. Your query is your location; the nearest points are the most relevant results. And just as you might filter for "open now" or re-check reviews before choosing, hybrid search and re-ranking refine the nearby candidates into the best answers.
Step by Step
Key Concepts
Nearest Neighbor Search
Finding the vectors closest to a query vector. Exact search checks all vectors (precise, slow); approximate nearest neighbor (ANN) checks a smart subset for near-instant results at scale.
Distance Metric
How similarity is measured — cosine similarity (angle), dot product, or Euclidean distance. The metric must match how the embedding model was trained for meaningful results.
Hybrid Search
Combining vector (semantic) search with keyword (lexical) search. It captures both meaning and exact terms, fixing pure vector search weakness on names, IDs, and rare tokens.
Re-ranking
A second stage that reorders the top candidates from fast retrieval using a more accurate (but slower) model, so the final top results are as relevant as possible.
Key Facts
- Vector search matches by meaning, so it finds relevant results that share no keywords with the query — the semantic advantage.
- Pure vector search can underperform on exact terms like product codes or names, which is why hybrid search (vector + keyword) is often best.
- A retrieve-then-re-rank pipeline is a common pattern: fast ANN retrieval for recall, then a precise re-ranker for the final ordering.
Real-World Applications
RAG retrieval
Vector search finds the document chunks most relevant to a users question, which are then fed to the LLM. Hybrid search and re-ranking improve which chunks are chosen, directly improving answer quality.
Site and product search
A search box uses vector search to understand intent (matching "waterproof jacket" to "rain coat") and hybrid search to still honour exact model numbers, giving results that feel smart and precise.
Frequently Asked Questions
What is vector search?
Vector search finds items whose meaning is most similar to a query by comparing embedding vectors, rather than matching exact keywords. The query is embedded into the same vector space as the stored content, and the search returns the nearest vectors — the semantically most relevant items. It can match related content even when the wording differs, which is why it underpins semantic search, RAG, and recommendation systems.
What is the difference between exact and approximate nearest neighbor search?
Exact nearest neighbor search compares the query against every stored vector to guarantee it finds the truly closest ones — accurate but slow, and impractical for large datasets. Approximate nearest neighbor (ANN) search uses specialised indexes to examine only a promising subset of vectors, returning near-best matches far faster. ANN trades a small, tunable amount of recall for orders-of-magnitude speed, which is what makes vector search scale to millions or billions of items.
What is hybrid search?
Hybrid search combines vector (semantic) search with traditional keyword (lexical) search and merges their results. Vector search captures meaning and handles synonyms and paraphrases, while keyword search excels at exact terms like names, product codes, and rare words that embeddings may blur. By combining both, hybrid search gets the recall of semantic matching and the precision of exact term matching, which usually produces better results than either alone.
Why add a re-ranking step to vector search?
Fast approximate retrieval is optimised for speed and recall, so it returns a good candidate set but may not order it perfectly by relevance. A re-ranking step takes those top candidates and reorders them using a more accurate but more expensive model — often a cross-encoder that examines the query and each candidate together. This retrieve-then-re-rank pattern trades a little extra latency for substantially better top results, which matters a lot for RAG answer quality and search precision.