# What Is an LLM Embedding? How AI Turns Text Into Vectors

> Learn how an LLM embedding transforms unstructured data into dense vectors to enable semantic search, RAG, and multimodal retrieval in production AI systems.

Source: https://camnangai.com/en/engineering/what-is-llm-embedding  
Published: 2026-08-10

An **LLM embedding** is a dense, lower-dimensional vector representation of data that enables
  computers to process semantic relationships mathematically.

While humans see words and images as discrete concepts, AI models need a numerical format to perform
operations. By mapping unstructured data into a continuous vector space, embeddings let you
calculate similarity between different inputs based on their underlying meaning rather than literal
matches.

An embedding transforms a piece of content — such as a sentence, a paragraph, or an image — into a
fixed-length list of floating-point numbers (a vector). These lists typically range from 300 to over
3,000 dimensions. Because these vectors sit in a multi-dimensional coordinate system based on their
context, items with similar meanings end up closer together. This mathematical proximity is the foundation for modern AI capabilities like semantic search and retrieval-augmented generation (RAG).

![An embedding illustrated: a passage of text turned into a list of numbers, then into points sitting close together in semantic space](https://camnangai.com/articles/embedding-la-gi/01-infographic-en.webp)

## What is an embedding?

Modern AI moved from sparse data representations to dense ones. In a sparse representation, such as
one-hot encoding, each item in a dataset is represented by a vector where only one entry is "1" and
all others are "0." For example, in a food-recommendation app with a 5,000-item dataset, "hot dog"
and "shawarma" would each be represented by a 5,000-long vector. Mathematically, these two vectors
are orthogonal, meaning the computer sees zero relationship between them, even though they are both
types of portable meat-based meals.

![Sparse one-hot encoding compared with a dense embedding, where similar foods cluster close together in a multi-dimensional space](https://camnangai.com/articles/embedding-la-gi/02-en.webp)

Sparse representations cause serious engineering problems. Large input vectors lead to a massive
number of weights in a neural network, requiring more data for effective training and exhausting
hardware memory limits. This makes it nearly impossible to scale systems or support on-device
machine learning (ODML). Dense vectors, or **embeddings**, solve this by reducing the
dimensionality. Instead of 5,000 dimensions for 5,000 items, the data is compressed into a
manageable space where every dimension is a floating-point number that captures latent features.

An embedding is a set of coordinates in a multi-dimensional space (typically 300, 768, or 1,536
dimensions) where location represents meaning. If "hot dog" is at one set of coordinates and
"shawarma" is at another, their proximity indicates semantic similarity. This dense structure allows
neural networks to train with fewer weights and perform more efficient calculations compared to
high-dimensional sparse data.

## Two different things called "embedding"

In AI infrastructure, "embedding" refers to both an internal model component and an external data output, which is a naming collision that costs teams more architecture arguments than it should. The "Embedding Layer" is an internal matrix of model weights used during training. This
layer is a **lookup table** where each word in a vocabulary is assigned a vector. During model
training, these numbers are updated via backpropagation as the model "learns" the optimal
representation for each word based on its context.

"Embedding Vectors," by contrast, are the external outputs generated by a pre-trained model acting
as an encoder. In this context, an engineer sends raw input — like a PDF or a search query — to the
model, which returns a fixed-length array of numbers. These arrays are then stored in vector
databases to enable efficient retrieval. The model acts as a black box that maps raw text into a
pre-defined vector space without further updating its internal weights.

![The embedding layer inside the model contrasted with the embedding vector that is stored in a vector database](https://camnangai.com/articles/embedding-la-gi/03-en.webp)

The training mechanics for these vectors often rely on architectures like Skipgram or Continuous Bag
of Words (CBOW). In Skipgram, the model uses a central target word to guess neighboring words within
a "sliding window." Through millions of these iterations, the model refines its internal lookup
table until words that appear in similar contexts — and so share similar meanings — have similar
vector values.

## How can a list of numbers carry meaning?

**Embeddings** capture meaning by treating dimensions as latent traits. A helpful analogy is
"personality embeddings." If you score a person on axes like introversion and extraversion, you can
represent them as a vector. While we cannot always manually interpret what "Dimension 42" represents
in a 1,536-dimension LLM embedding, the model has determined that similar concepts will share
similar values at that position.

![How Skipgram and CBOW learn meaning from surrounding words, alongside the vector arithmetic King minus Man plus Woman approximately equals Queen](https://camnangai.com/articles/embedding-la-gi/04-en.webp)

This mathematical representation allows vector arithmetic, famously shown by the formula:
`King - Man + Woman = Queen`. Subtract the "manhood" vector from "king," add "womanhood," and the
resulting coordinates land nearest to the vector for "queen." The model has captured abstract
concepts like gender and royalty as directions in space.

To determine how similar two pieces of content are, engineers use **cosine similarity**. This
measures the cosine of the angle between two vectors, returning a score between -1 and 1.0. A score of 1.0 means the vectors point in the same direction, which is maximum semantic similarity. In Python, the
calculation looks like this:

```python
def cosine_similarity(a, b):
    dot_product = sum(x * y for x, y in zip(a, b))
    magnitude_a = sum(x * x for x in a) ** 0.5
    magnitude_b = sum(x * x for x in b) ** 0.5
    return dot_product / (magnitude_a * magnitude_b)
```

This enables "vibes-based search," where a query for "backups" can find documents containing
"recovery" or "redundancy" even without a literal keyword match.

## What are embeddings used for?

Embeddings are a versatile tool for managing unstructured data. Four primary engineering use cases:

1.  **Related content:** Engineers can build "related articles" features by pre-calculating
    embeddings for every entry in a database. When a user views an article, the system runs a cosine
    similarity check to find the top 10 nearest neighbors. Simon Willison uses this pattern in SQLite
    to automatically link related blog posts.
2.  **Semantic search:** Traditional search relies on keyword matching (lexical search).
    Embedding-based search, such as the "Faucet Finder" project, lets users find products based
    on visual or conceptual similarity. A user can find a cheaper faucet that looks like a luxury
    model because the image embeddings occupy a similar mathematical neighborhood.
3.  **Clustering:** Tools like scikit-learn can use embeddings to group thousands of data points into
    themes automatically. For instance, an engineer can take 10,000 GitHub issues, generate their
    embeddings, and cluster them into categories like "UI Bugs" or "Security" without manual tagging.
4.  **[Retrieval-Augmented Generation (RAG)](https://camnangai.com/en/knowledge/what-is-rag):** RAG is a "cheap trick" to
    answer questions about private data without retraining a model. When a user asks a question, the
    system finds the most relevant excerpts from a private database via embeddings and pastes them
    into an [LLM](https://camnangai.com/en/knowledge/what-is-an-llm) prompt as context. This lets the LLM give
    accurate answers based on data it was never trained on.

## Beyond text: multimodal embeddings

Modern models have moved toward a "shared vector space" where different types of data are mapped to
the same coordinates. Multimodal models like CLIP or Gemini Embedding 2 can place an image of a dog
and the word "dog" in the same mathematical neighborhood. This enables cross-modal search, where a
text query can retrieve the most semantically relevant image, video, or audio file.

![Text, image, video and audio mapped into one shared vector space, with the modality gap between two clusters](https://camnangai.com/articles/embedding-la-gi/05-en.webp)

A critical metric in multimodal retrieval is the **modality gap**. This is the L2 distance between
the cluster of text embeddings and the cluster of image embeddings. Models like Qwen3-VL-2B have a
smaller gap (0.25) compared to others like Gemini 2 (0.73). A smaller gap generally makes cross-modal similarity search more reliable. The distance between 0.25 and 0.73 reads like a rounding difference, but in retrieval it separates finding the right image from finding something that merely looks adjacent. Advanced models like Gemini 2 now support a wide range
of modalities, including:

- Text and PDF
- Image and Video
- Audio

## Choosing an embedding model: dimensions, cost, and quality

Selecting a model requires balancing storage costs against retrieval quality. Recent benchmarks give
you a technical basis for that choice:

**Dimension compression:** Matryoshka Representation Learning (MRL) lets an engineer truncate a long
vector (e.g., 3072 dimensions) down to a much smaller size (e.g., 256) while retaining most of its
accuracy. Truncating from 3072 to 256 gives a 12x storage reduction. For a collection of 100 million vectors at float32, this reduces the infrastructure requirement from 1.14 TB to 95 GB. If I had to design around a single number in this section, it would be that 1.14 TB, because storage is the constraint that turns up on an invoice every month. Voyage
Multimodal 3.5 and Jina Embeddings v4 lead this category, because both were trained with MRL as an explicit objective, and at 256 dimensions Voyage loses only 0.7% of its quality.

**Quantization and hardware:** Vector precision affects storage and search speed. While
FP16/bfloat16 offers a 2x storage reduction with zero quality loss, engineers must watch out for
INT8 "gotchas." INT8 quantization can speed up CPU inference by 2.7–3.4x, but it is actually 4-5x
**slower** than FP32 on GPUs. For extreme efficiency, binary quantization offers 32x memory
reduction; systems can perform ~1 billion **Hamming distance** calculations per second with binary
vectors, which is 7x faster than prenormalized angular distance.

**Model performance:** In the CCKM benchmark, **Gemini 2** emerged as the top generalist for
cross-lingual and long-document tasks. **Qwen3-VL-2B** outperformed closed-source APIs in
cross-modal precision, making it the preferred choice for text-to-image pipelines.

![Matryoshka-style dimension compression: a 3072-dimension vector truncated down to 256 dimensions, collapsing storage cost](https://camnangai.com/articles/embedding-la-gi/06-en.webp)

## What embeddings cannot do

Embeddings have clear technical limits:

- **[Context window](https://camnangai.com/en/knowledge/context-window) limits:** Models have a hard truncation point.
  Gemini Embedding 2 held its retrieval accuracy across documents all the way up to 32K characters,
  but many lightweight models like _mxbai-embed-large_ cap out at a strict **512-token context
  window** ([tokens](https://camnangai.com/en/knowledge/what-are-ai-tokens), not characters). Feed one of those a
  4,000-character document and accuracy tanks, because most of the data is discarded.
- **Domain gap:** A model trained on general web data may fail on medical, legal, or niche technical
  jargon. Without fine-tuning on a domain-specific corpus, the model may place "atrial fibrillation"
  and "heart attack" in the same neighborhood despite their clinical differences.
- **Lack of reasoning:** Embeddings are for finding candidate documents, not for "thinking." They
  identify relevance based on proximity, but they do not reason over the data. Production-grade
  systems often rank in phases: embeddings for broad retrieval, then a cross-encoder "re-ranker" to
  handle final accuracy and logic.

## When should you use embeddings?

Use embeddings whenever a project involves unstructured data — such as images, video, or long-form
text — where traditional lexical search (BM25) fails to capture intent. They are essential for
building discovery engines, recommendation systems, and RAG pipelines that must scale across
modalities.

But engineers should not rely only on public leaderboards. Because the "best" model depends on your
specific data types and hardware constraints (CPU vs. GPU), the most durable investment is your own
evaluation pipeline. Test new models against your own corpus to find the optimal tradeoff between dimension size, storage cost, and retrieval latency. A leaderboard has never seen your data.

## References

- [Embeddings | Machine Learning | Google for Developers](https://developers.google.com/machine-learning/crash-course/embeddings)
- [Embeddings: What they are and why they matter](https://simonwillison.net/2023/Oct/23/embeddings/)
- [The Illustrated Word2vec — Jay Alammar](https://jalammar.github.io/illustrated-word2vec/)
- [Vector embeddings | OpenAI API](https://developers.openai.com/api/docs/guides/embeddings)
- [Embedding Tradeoffs, Quantified | Vespa Blog](https://blog.vespa.ai/embedding-tradeoffs-quantified/)
- [Best Embedding Model for RAG 2026: 10 Models Compared — Milvus Blog](https://milvus.io/blog/choose-embedding-model-rag-2026.md)
- [Gemini Embedding 2 is now generally available](https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-embedding-2-generally-available/)
- [On the Theoretical Limitations of Embedding-Based Retrieval](https://arxiv.org/abs/2508.21038)
- [What are Vector Embeddings | Pinecone](https://www.pinecone.io/learn/vector-embeddings/)