Skip to content

What are Knowledge Graphs and GraphRAG? Graph Retrieval Architecture

GraphRAG combines vector similarity search with structured knowledge graphs to enable multi-hop reasoning and holistic data summaries for LLMs.

Tuan Tran Van
9 min read
Contents (7 sections)
  1. The Limits of Baseline Vector RAG: Why Similarity Search Falls Short
  2. Knowledge Graphs: Structuring Unstructured Data into Entities and Relationships
  3. The Microsoft GraphRAG Architecture: Indexing from Extraction to Leiden Communities
  4. Dual Retrieval Paradigms: Local Search vs Global Search
  5. Vector RAG vs GraphRAG: Trade-offs in Cost, Latency, and Complexity
  6. When Should an AI Engineer Invest in GraphRAG?
  7. References

GraphRAG is a retrieval paradigm that combines vector similarity search over text chunks with graph-based structured representations—including entity nodes, relationship edges, and hierarchical community structures—to supply (LLMs) with contextually grounded, relational data for response synthesis. By organizing unstructured private corpora into explicit property graphs and community hierarchies, you enable language models to query both localized topological neighborhoods and macro-level dataset clusters.

In contrast to naive top-$k$ similarity lookups that extract isolated text fragments based solely on vector distance in embedding space, GraphRAG captures explicit entity relationships and topological dataset structures. This hybrid retrieval architecture enables you to execute multi-hop query traversals and global thematic summaries across complex document collections while maintaining direct, auditable evidence citations back to the source text.

Illustration of GraphRAG architecture combining knowledge graphs with LLMs for connected reasoning

The Limits of Baseline Vector RAG: Why Similarity Search Falls Short

Baseline Retrieval-Augmented Generation () relies on slicing source documents into isolated text chunks, those fragments into high-dimensional vector representations, and executing top-$k$ vector similarity search against user queries. While effective for simple semantic matching over localized passages, this architecture encounters fundamental bottlenecks when queries require connecting disparate information across multiple documents or understanding high-level corpus themes.

Vector similarity search routinely fails at "connecting the dots"—questions that require traversing related attributes across separate entities. For instance, testing baseline RAG on the Violent Incident Information from News Articles (VIINA) dataset reveals severe architectural limits: when queried with "What has Novorossiya done?", baseline vector search retrieves text fragments that fail to contain explicit context about Novorossiya's actions, leading the LLM to report that the text provides no specific information. In contrast, graph-based retrieval traces relational edges across multiple document segments to extract specific linked target entities, such as plans to target Roshen, the Odessa Canning Factory, the Odessa Regional Radio Television Transmission Center, the National Television Company of Ukraine, and blowing up ATMs of PrivatBank.

Two core bottlenecks of baseline vector RAG: connecting disparate entities and whole-dataset summarization

Furthermore, baseline RAG fails on holistic, corpus-wide queries such as "What are the top 5 themes in the data?". Because vector search matches semantically similar keywords (e.g., retrieving chunks containing the word "theme"), it populates the LLM with arbitrary text fragments—such as urban development, Vladimir Putin's meeting with UAE President Mohammed bin Zayed Al Nahyan, or regional investment climate ratings—rather than synthesizing global dataset structures. Beyond extraction failures, vector embeddings suffer from an explainability black-box issue, offering no auditable mechanism to verify why a text chunk was retrieved compared to explicit, inspectable graph edges.

Knowledge Graphs: Structuring Unstructured Data into Entities and Relationships

Knowledge graphs structure unstructured source text into labeled property graphs composed of entity nodes, typed relationships, and metadata attributes linked back to raw text chunks. Entity nodes represent categorized real-world concepts (such as people, organizations, or locations), while directed edges capture explicitly typed relationships between nodes (such as MEMBER_OF or TARGETS). This graph model serves as a flexible memory layer alongside the capabilities of LLMs.

Property Graph structure comprising entity nodes, directed relationship edges, and metadata properties

Converting raw text into a property graph involves executing dedicated extractors over chunked input units. Frameworks like LlamaIndex provide several extraction primitives: SimpleLLMPathExtractor parses single-hop (subject, predicate, object) triples directly from text using basic prompts; DynamicLLMPathExtractor guides extraction using optional target lists of allowed entity types and relations; and SchemaLLMPathExtractor enforces strict dynamic validation schemas using Pydantic to ensure triples strictly conform to allowed entity-relation pairs. Additionally, ImplicitPathExtractor parses existing node.relationships attributes attached to text nodes without incurring extra LLM costs. If no extractors are explicitly passed to PropertyGraphIndex.from_documents, LlamaIndex defaults to running both SimpleLLMPathExtractor and ImplicitPathExtractor concurrently.

You can build a property graph index using LlamaIndex primitives by instantiating graph extractors over your documents as shown in the Python configuration below:

python
from llama_index.core import PropertyGraphIndex
from llama_index.core.indices.property_graph import SimpleLLMPathExtractor
 
# Extracting entity-relation triples from text nodes
kg_extractor = SimpleLLMPathExtractor(
    llm=llm,
    max_paths_per_chunk=10,
    num_workers=4
)
 
index = PropertyGraphIndex.from_documents(
    documents,
    kg_extractors=[kg_extractor]
)

The Microsoft GraphRAG Architecture: Indexing from Extraction to Leiden Communities

Microsoft Research's GraphRAG indexing pipeline establishes a hierarchical graph workflow designed to process private dataset collections. The process begins by slicing raw corpus data into standardized TextUnits. These TextUnits act as granular, analyzable reference objects throughout the pipeline and serve as permanent provenance anchors for generated outputs.

Once text is partitioned, an LLM scans all TextUnits to extract all entities (such as individuals, places, and organizations), relationships, and key claims. To enable multi-level reasoning, GraphRAG applies the Leiden graph algorithm to perform hierarchical community detection across the knowledge graph. In topological visualizations of this structure, entity node sizes represent their graph degree (number of connections), while distinct color partitions denote community memberships.

Microsoft GraphRAG indexing pipeline from TextUnits to Leiden community detection and multi-level summaries

After clustering the graph into semantic communities, the pipeline performs bottom-up pre-summarization. An LLM generates dedicated summary reports for every community layer from the bottom up, capturing the central concepts, constituent entities, and underlying relationships. These pre-summarized community reports allow the system to answer complex prompts across varying levels of abstraction without overflowing the model's context window.

Graph retrieval architectures execute distinct search paradigms depending on whether the query targets explicit entity details or dataset-wide concepts. Query execution engines route requests to Local Search, Global Search, or hybrid execution mechanisms to optimize context delivery to the LLM.

Contrasting Local Search topological neighbor traversal with Global Search Map-Reduce execution

Local Search targets explicit, entity-focused questions. It functions by identifying entity nodes mentioned within the user prompt, fanning out to immediate topological neighbors (typically 1 to 3 hops deep), and retrieving linked text chunks along with entity-relationship pairs. This creates a grounded context window complete with explicit source citations. Conversely, Global Search bypasses vector chunk retrieval entirely when answering holistic queries (e.g., summarizing top dataset themes). Instead, Global Search iterates across pre-summarized Leiden community reports, synthesizing macro-level insights across dataset clusters without risking context window saturation.

To resolve queries requiring both fine-grained entity details and broad contextual framing, hybrid paradigms like DRIFT search combine localized topological expansion (1–3 hop neighbor traversal) with high-level Leiden community summaries. In enterprise graph environments like Amazon Neptune, dual execution models use KnowledgeGraphRAGRetriever with with_nl2graphquery=True and a graph_traversal_depth of 3 hops. This pairs graph neighborhood extraction with LLM-generated openCypher queries (NL2CYPHER_PROMPT). Using response_mode="refine" in LlamaIndex's RetrieverQueryEngine forces the LLM to independently execute and synthesize reasoning over both structured graph traversal and openCypher results, mitigating Cypher syntax errors and empty result sets.

For vector stores containing document metadata, you can execute graph-based multi-hop traversal without deploying a dedicated graph database. Using LangChain's GraphRetriever, metadata fields serve as graph edges across stores like AstraDB, Chroma, or InMemoryVectorStore:

python
# Traversing metadata relationships to retrieve neighbor context
from graph_retriever.strategies import Eager
from langchain_graph_retriever import GraphRetriever
 
traversal_retriever = GraphRetriever(
    store=vector_store,
    edges=[("habitat", "habitat"), ("origin", "origin")],
    strategy=Eager(k=5, start_k=1, max_depth=2)
)

In this setup, start_k=1 selects the single nearest vector match as the seed node, max_depth=2 restricts topological exploration to nodes within two metadata relationship hops (sharing habitat or origin), and k=5 caps the total retrieved document count.

Vector RAG vs GraphRAG: Trade-offs in Cost, Latency, and Complexity

Transitioning from standard vector retrieval to GraphRAG introduces structural engineering trade-offs across indexing cost, query latency, system complexity, and output precision. Standard vector RAG requires a single embedding pass per text chunk, resulting in rapid ingestion and low upfront compute overhead. GraphRAG requires substantial offline compute during indexing due to multiple LLM extraction passes across every TextUnit, entity resolution steps, and Leiden hierarchical clustering algorithms.

At query time, standard vector RAG executes a single $k$-NN vector lookup, maintaining low retrieval latency. GraphRAG queries involve graph database traversals, topological expansions, or iterating over community summary reports, which increases overall response latency. However, GraphRAG significantly outpaces standard vector search in multi-hop accuracy and reduction.

To evaluate these systems beyond raw retrieval, Microsoft Research established a combination of absolute and qualitative benchmarks. Absolute faithfulness is measured via SelfCheckGPT to verify that generated assertions are strictly grounded in source context. Qualitative pairwise performance is evaluated by LLM graders across three core metrics: comprehensiveness (completeness of the response within the implied context), human enfranchisement (provision of supporting source citations), and diversity (provision of differing viewpoints, such as contrasting UNIAN and RIA news perspectives within the VIINA dataset).

ParameterBaseline Vector RAGGraphRAG
Data Indexing CostLow (single embedding pass per chunk)High (multi-pass LLM extraction, Leiden clustering)
Multi-hop Query AccuracyLow (fails to connect disparate text attributes)High (traverses 1–3+ hop relational edges)
Global Theme SynthesisPoor (retrieves irrelevant chunks via keyword matching)High (iterates over pre-summarized Leiden communities)
Query LatencyLow (single $k$-NN vector lookup)Moderate to High (graph traversals & community summaries)
Provenance ExplainabilityLow (opaque vector distance metrics)High (explicit entity edges and source text citations)

When Should an AI Engineer Invest in GraphRAG?

You should stick with baseline vector RAG when your workload consists of basic semantic lookups over flat, unstructured documents, or when operational constraints dictate minimal indexing compute and low query latency. Baseline vector search is sufficient for isolated chunk retrieval where cross-document entity relationships and macro thematic summaries are unnecessary.

You should transition to GraphRAG when engineering enterprise systems—such as Customer 360 platforms, narrative intelligence feeds, or complex domain ontologies—that demand explicit multi-hop relationship tracking, global thematic analysis over massive private corpora, and fully auditable evidence provenance.

References

Check yourself

Three quick questions to see whether the main ideas stuck.

  1. Where does baseline vector RAG experience its most severe failure mode?
  2. What is the primary function of the Leiden algorithm in Microsoft GraphRAG?
  3. Which search mechanism does GraphRAG use for corpus-wide thematic inquiries?

Share this article

AI glossary