What Is an LLM Embedding? How AI Turns Text Into Vectors

An LLM embedding is a dense, lower-dimensional vector representation of data that enables computers to process semantic relationships mathematically.

While humans see words and images as discrete concepts, AI models need a numerical format to perform operations. By mapping unstructured data into a continuous vector space, embeddings let you calculate similarity between different inputs based on their underlying meaning rather than literal matches.

An embedding transforms a piece of content — such as a sentence, a paragraph, or an image — into a fixed-length list of floating-point numbers (a vector). These lists typically range from 300 to over 3,000 dimensions. Because these vectors sit in a multi-dimensional coordinate system based on their context, items with similar meanings end up closer together. This mathematical proximity is the foundation for modern AI capabilities like semantic search and retrieval-augmented generation (RAG).

An embedding illustrated: a passage of text turned into a list of numbers, then into points sitting close together in semantic space

What is an embedding?

Modern AI moved from sparse data representations to dense ones. In a sparse representation, such as one-hot encoding, each item in a dataset is represented by a vector where only one entry is "1" and all others are "0." For example, in a food-recommendation app with a 5,000-item dataset, "hot dog" and "shawarma" would each be represented by a 5,000-long vector. Mathematically, these two vectors are orthogonal, meaning the computer sees zero relationship between them, even though they are both types of portable meat-based meals.

Sparse one-hot encoding compared with a dense embedding, where similar foods cluster close together in a multi-dimensional space

Sparse representations cause serious engineering problems. Large input vectors lead to a massive number of weights in a neural network, requiring more data for effective training and exhausting hardware memory limits. This makes it nearly impossible to scale systems or support on-device machine learning (ODML). Dense vectors, or embeddings, solve this by reducing the dimensionality. Instead of 5,000 dimensions for 5,000 items, the data is compressed into a manageable space where every dimension is a floating-point number that captures latent features.

An embedding is a set of coordinates in a multi-dimensional space (typically 300, 768, or 1,536 dimensions) where location represents meaning. If "hot dog" is at one set of coordinates and "shawarma" is at another, their proximity indicates semantic similarity. This dense structure allows neural networks to train with fewer weights and perform more efficient calculations compared to high-dimensional sparse data.

Two different things called "embedding"

In AI infrastructure, "embedding" refers to both an internal model component and an external data output. The "Embedding Layer" is an internal matrix of model weights used during training. This layer is a lookup table where each word in a vocabulary is assigned a vector. During model training, these numbers are updated via backpropagation as the model "learns" the optimal representation for each word based on its context.

"Embedding Vectors," by contrast, are the external outputs generated by a pre-trained model acting as an encoder. In this context, an engineer sends raw input — like a PDF or a search query — to the model, which returns a fixed-length array of numbers. These arrays are then stored in vector databases to enable efficient retrieval. The model acts as a black box that maps raw text into a pre-defined vector space without further updating its internal weights.

The embedding layer inside the model contrasted with the embedding vector that is stored in a vector database

The training mechanics for these vectors often rely on architectures like Skipgram or Continuous Bag of Words (CBOW). In Skipgram, the model uses a central target word to guess neighboring words within a "sliding window." Through millions of these iterations, the model refines its internal lookup table until words that appear in similar contexts — and so share similar meanings — have similar vector values.

How can a list of numbers carry meaning?

Embeddings capture meaning by treating dimensions as latent traits. A helpful analogy is "personality embeddings." If you score a person on axes like introversion and extraversion, you can represent them as a vector. While we cannot always manually interpret what "Dimension 42" represents in a 1,536-dimension LLM embedding, the model has determined that similar concepts will share similar values at that position.

How Skipgram and CBOW learn meaning from surrounding words, alongside the vector arithmetic King minus Man plus Woman approximately equals Queen

This mathematical representation allows vector arithmetic, famously shown by the formula: King - Man + Woman = Queen. Subtract the "manhood" vector from "king," add "womanhood," and the resulting coordinates land nearest to the vector for "queen." The model has captured abstract concepts like gender and royalty as directions in space.

To determine how similar two pieces of content are, engineers use cosine similarity. This measures the cosine of the angle between two vectors, returning a score between -1 and 1.0. A score of 1.0 means the vectors point in the same direction — maximum semantic similarity. In Python, the calculation looks like this:

python

def cosine_similarity(a, b):
    dot_product = sum(x * y for x, y in zip(a, b))
    magnitude_a = sum(x * x for x in a) ** 0.5
    magnitude_b = sum(x * x for x in b) ** 0.5
    return dot_product / (magnitude_a * magnitude_b)

This enables "vibes-based search," where a query for "backups" can find documents containing "recovery" or "redundancy" even without a literal keyword match.

What are embeddings used for?

Embeddings are a versatile tool for managing unstructured data. Four primary engineering use cases:

Related content: Engineers can build "related articles" features by pre-calculating embeddings for every entry in a database. When a user views an article, the system runs a cosine similarity check to find the top 10 nearest neighbors. Simon Willison uses this pattern in SQLite to automatically link related blog posts.
Semantic search: Traditional search relies on keyword matching (lexical search). Embedding-based search, such as the "Faucet Finder" project, lets users find products based on visual or conceptual similarity. A user can find a cheaper faucet that looks like a luxury model because the image embeddings occupy a similar mathematical neighborhood.
Clustering: Tools like scikit-learn can use embeddings to group thousands of data points into themes automatically. For instance, an engineer can take 10,000 GitHub issues, generate their embeddings, and cluster them into categories like "UI Bugs" or "Security" without manual tagging.
Retrieval-Augmented Generation (RAG): RAG is a "cheap trick" to answer questions about private data without retraining a model. When a user asks a question, the system finds the most relevant excerpts from a private database via embeddings and pastes them into an LLM prompt as context. This lets the LLM give accurate answers based on data it was never trained on.

Beyond text: multimodal embeddings

Modern models have moved toward a "shared vector space" where different types of data are mapped to the same coordinates. Multimodal models like CLIP or Gemini Embedding 2 can place an image of a dog and the word "dog" in the same mathematical neighborhood. This enables cross-modal search, where a text query can retrieve the most semantically relevant image, video, or audio file.

Text, image, video and audio mapped into one shared vector space, with the modality gap between two clusters

A critical metric in multimodal retrieval is the modality gap. This is the L2 distance between the cluster of text embeddings and the cluster of image embeddings. Models like Qwen3-VL-2B have a smaller gap (0.25) compared to others like Gemini 2 (0.73). A smaller gap generally makes cross-modal similarity search more reliable. Advanced models like Gemini 2 now support a wide range of modalities, including:

Text and PDF
Image and Video
Audio

Choosing an embedding model: dimensions, cost, and quality

Selecting a model requires balancing storage costs against retrieval quality. Recent benchmarks give you a technical basis for that choice:

Dimension compression: Matryoshka Representation Learning (MRL) lets an engineer truncate a long vector (e.g., 3072 dimensions) down to a much smaller size (e.g., 256) while retaining most of its accuracy. Truncating from 3072 to 256 gives a 12x storage reduction. For a collection of 100 million vectors at float32, this reduces the infrastructure requirement from 1.14 TB to 95 GB. Voyage Multimodal 3.5 and Jina Embeddings v4 lead this category, because both were trained with MRL as an explicit objective — at 256 dimensions Voyage loses only 0.7% of its quality.

Quantization and hardware: Vector precision affects storage and search speed. While FP16/bfloat16 offers a 2x storage reduction with zero quality loss, engineers must watch out for INT8 "gotchas." INT8 quantization can speed up CPU inference by 2.7–3.4x, but it is actually 4-5x slower than FP32 on GPUs. For extreme efficiency, binary quantization offers 32x memory reduction; systems can perform ~1 billion Hamming distance calculations per second with binary vectors, which is 7x faster than prenormalized angular distance.

Model performance: In the CCKM benchmark, Gemini 2 emerged as the top generalist for cross-lingual and long-document tasks. Qwen3-VL-2B outperformed closed-source APIs in cross-modal precision, making it the preferred choice for text-to-image pipelines.

Matryoshka-style dimension compression: a 3072-dimension vector truncated down to 256 dimensions, collapsing storage cost

What embeddings cannot do

Embeddings have clear technical limits:

Context window limits: Models have a hard truncation point. Gemini Embedding 2 held its retrieval accuracy across documents all the way up to 32K characters, but many lightweight models like mxbai-embed-large cap out at a strict 512-token context window (tokens, not characters). Feed one of those a 4,000-character document and accuracy tanks, because most of the data is discarded.
Domain gap: A model trained on general web data may fail on medical, legal, or niche technical jargon. Without fine-tuning on a domain-specific corpus, the model may place "atrial fibrillation" and "heart attack" in the same neighborhood despite their clinical differences.
Lack of reasoning: Embeddings are for finding candidate documents, not for "thinking." They identify relevance based on proximity, but they do not reason over the data. Production-grade systems often rank in phases: embeddings for broad retrieval, then a cross-encoder "re-ranker" to handle final accuracy and logic.

When should you use embeddings?

Use embeddings whenever a project involves unstructured data — such as images, video, or long-form text — where traditional lexical search (BM25) fails to capture intent. They are essential for building discovery engines, recommendation systems, and RAG pipelines that must scale across modalities.

But engineers should not rely only on public leaderboards. Because the "best" model depends on your specific data types and hardware constraints (CPU vs. GPU), the most durable investment is your own evaluation pipeline. Test new models against your own corpus to find the optimal tradeoff between dimension size, storage cost, and retrieval latency.