Mathematical definition
Cosine similarity between two vectors A and B in Euclidean space is the cosine of the angle θ between them. Formally:
A · B Σᵢ AᵢBᵢ
similarity(A,B) = ------- = ----------------------------
‖A‖ ‖B‖ √(Σᵢ Aᵢ²) · √(Σᵢ Bᵢ²)That is: inner product of the vectors divided by the product of their L2 norms. In practice: it is the inner product (dot product) of normalized vectors. When vectors are already normalized (L2 norm = 1), cosine similarity reduces to the dot product:
import numpy as np
def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
"""Cosine similarity: value in [-1, 1]. 1 = identical, 0 = orthogonal, -1 = opposite."""
a_norm = a / np.linalg.norm(a)
b_norm = b / np.linalg.norm(b)
return float(np.dot(a_norm, b_norm))- 1: same direction (maximum similarity)
- 0: orthogonal (no relation)
- -1: opposite directions
For text embeddings, values usually lie in [0, 1] because models produce vectors in semi-spaces where angles > 90° are rare.
Embeddings and semantic space
LLMs and embedding models (Sentence-BERT, OpenAI Embeddings, Cohere, etc.) map text to dense vectors in R^n (typically n = 384, 768, 1536 or more). In this space:
- Semantically similar texts are geometrically close
- Cosine similarity (or dot product with normalized vectors) measures that closeness
So: finding “similar texts” becomes finding “vectors with high cosine”.
Why cosine similarity helps LLMs
1. Retrieval (RAG) and relevant context
In RAG (Retrieval-Augmented Generation), the flow is:
- Index: convert documents into embeddings and store (e.g. vector DB).
- Query: convert the user’s question into an embedding.
- Retrieve: fetch the k vectors most similar to the query embedding (using cosine or dot product).
- Generate: inject those chunks as context into the LLM prompt.
If retrieval returns irrelevant chunks, the LLM tends to “hallucinate” or drift. If it returns chunks highly similar to the question, the context is relevant and the answer tends to be more accurate and grounded. Cosine similarity is the metric that ranks which chunks are most relevant.
2. Independence of magnitude (norm)
Unlike Euclidean distance, cosine similarity does not depend on vector length, only on direction. That is useful because:
- Long documents yield vectors with larger norm; with Euclidean distance, a long doc could always be “farther”.
- With cosine, a short paragraph and a long document can be equally similar to the query if their semantic content is aligned.
3. Efficiency in production
With normalized vectors (L2 norm = 1), cosine similarity = dot product. In many vector DBs (Pinecone, Weaviate, pgvector, etc.):
- Indexes like IVF or HNSW are optimized for nearest neighbor by dot product or cosine.
- Search is sub-linear in the number of vectors (no need to compare to all).
This allows scaling RAG to millions of chunks without degrading latency.
Typical RAG implementation
# Pseudocode for retrieval with cosine similarity
def retrieve(query: str, top_k: int = 5) -> list[Chunk]:
query_embedding = embedding_model.encode(query, normalize_embeddings=True)
# Vector DB returns top_k by cosine similarity (or dot product)
results = vector_db.search(query_embedding, top_k=top_k, metric="cosine")
return [hit.chunk for hit in results]Normalizing at index and query time ensures you are actually using cosine (dot product on unit vectors).
Summary
| Concept | Role |
|---|---|
| Cosine similarity | Measures semantic alignment between vectors (query embedding vs. chunk embeddings). |
| RAG | Uses this metric to choose the most relevant context before calling the LLM. |
| Effect on LLMs | Less hallucination and more accurate answers when retrieved context has high similarity to the question. |
The better the retrieval (and thus the use of cosine similarity), the better the LLM tends to perform on knowledge-based tasks (QA, support, documentation).