What is LangSmith? Observability and Evals for LangChain Agents

LangSmith is a framework-agnostic agent engineering platform built around three pillars: observability, evaluation, and deployment.

You use it to handle the non-deterministic behavior of large language models (LLMs) by tracing and inspecting every intermediate step in the system's reasoning path.

The platform lets you see inside the black box of an AI agent, from early prompt experiments to production monitoring.

Instead of guessing why an agent went wrong, you get structured data on retrieval-augmented generation (RAG) flows, tool calls, and reasoning logic — enough to tune performance and keep costs under control.

Seeing inside the black box of a running AI agent, with each reasoning step surfaced as observable data

What is LangSmith?

LangSmith is a framework-agnostic platform. It integrates most deeply with the LangChain ecosystem, but you can instrument custom agent systems through the Python and JS/TS SDKs, or through OpenTelemetry. It is the control layer for an agent's whole development lifecycle, not a logging library.

The agent development lifecycle in LangSmith — build, test, deploy, monitor, improve — alongside the three pillars of observability, evaluation, and deployment

Across that lifecycle, LangSmith shows up at every stage: the Playground for testing prompts and seeing results immediately, tracing for recording complex execution paths, datasets for building test suites out of real traffic, and monitoring once the system is in production. You can break apart each node in a LangGraph graph, or each execution step of a Deep Agent.

For a systems engineer, the real value is being able to measure what changes when you swap a model or rewrite the instructions in a prompt. You get concrete numbers on latency, cost, and output quality to show the new version beats the old one, rather than relying on trial and error.

Traces, runs, threads and projects: what LangSmith records

To keep observability data manageable, LangSmith organizes it in a hierarchy. A run is the atomic unit of work — a single model call, tool invocation, or vector DB retrieval — and maps to a span in OpenTelemetry terms. A trace is a tree of runs covering one user-facing operation. Each trace is capped at 25,000 runs; past that, the system stops recording.

The nested shape of observability data: many runs inside one trace, many traces linked into a thread, with a trajectory as the flat view of the same data

A thread is a sequence of traces linked by a thread_id, representing one multi-turn session. Where a thread preserves the nested structure, a trajectory gives you a flat, time-ordered list of AI, Human, and Tool messages. Trajectories are faster to read because they strip the noise of intermediate execution steps; threads keep the full technical detail you need for debugging.

All of this lives inside projects, which are separate containers per service or per environment (dev, staging, prod).

Concept	Shape	What it holds	What you use it for
Run	Single unit	Input, output, metadata for one step	Inspecting one specific operation
Trace	Tree of runs	Every step within one operation	Debugging logic failures or latency
Thread	Sequence of traces	Multi-turn conversation history	Watching agent behavior across a session
Trajectory	Flat message list	AI, Human, and Tool messages in time order	Reading the exchange quickly, less noise

You enrich this data with feedback, tags, and metadata. Feedback attaches scores to individual runs — the basis for both human review and automated evaluation. Tags are strings for high-level filtering, and metadata holds key-value context such as application version or environment.

Evaluating quality: datasets, experiments and LLM-as-judge

Evaluation in LangSmith splits into offline and online work. Offline evaluation happens during development: you benchmark model versions and run regression tests against datasets, which are collections of examples pairing an input with a reference output. Running your application against a dataset produces an experiment, capturing outputs and scores so you can compare configurations side by side.

The evaluation loop: a failing production trace is added to a dataset, an experiment runs against it, then the old and new versions are compared

Online evaluation covers production monitoring. You have no reference outputs for live traffic, so you lean on heuristics, safety checks, and reference-free LLM-as-judge evaluators. They watch runs and threads in real time and flag quality degradation before it reaches users at scale. The loop closes when an issue caught online becomes a new example in your offline dataset.

The LLM-as-judge pattern uses one model to grade another's output against a rubric encoded in a prompt:

Reference-free evaluators assess clarity, tone, or safety-policy adherence with no expected answer.
Reference-based evaluators compare output against a ground-truth reference to check factual accuracy.

For subjective work like summarization, pairwise evaluation — where a human or LLM judge picks the better of two outputs — often beats assigning absolute scores. To scale human review, annotation queues give reviewers a structured workflow for scoring traces against a rubric.

Context Hub: managing prompts and context like code

Instructions, policies, and examples change far faster than application code. Context Hub gives you a central place to store and version the non-code parts of your agents — individual prompts (message templates) and contexts (bundles of instructions, tools, and policies). Decoupling them from your core logic lets domain experts change behavior without a code deploy.

Context Hub tags each context version dev, staging, or prod, so changing an agent's behavior means moving a tag rather than redeploying code

It behaves like version control for agent logic. You tag versions dev, staging, or prod so each agent pulls the right configuration for its environment, and teammates can comment on changes and review history before promoting a version. That matters most in multi-agent systems where several agents share a set of skills or instructions.

Context Hub is also where the files that define agent behavior live — AGENTS.md files, skill sets, and project-specific examples. It connects to the LangSmith Playground, so you can test a prompt or context bundle against different models before committing it. Context gets the same governance and auditability you expect from source code.

LangSmith Engine and SmithDB: what changed after Interrupt 2026

At Interrupt 2026, LangChain shipped two upgrades aimed at the scale of modern agent workloads.

SmithDB is a purpose-built database for agent observability, written in Rust on top of Apache DataFusion and Vortex. It stores durable trace data in object storage and keeps ingestion and query services stateless, which is what makes deeply nested spans and long-running agent operations workable:

Full trace tree load: 92ms (P50).
Single run load: 71ms (P50).
Up to 15x faster than the previous backend, while keeping the system stateless and easy to scale.

LangSmith Engine is the autonomous layer on top. It watches production data, clusters failures into prioritized issues, and diagnoses root causes against your code and prompts. Once enabled, it can propose fixes by opening pull requests or by creating online evaluators that catch the same regression next time. Your role moves from manual debugging to reviewing and merging suggestions.

The LangSmith Engine loop: watch production traces, cluster failures, diagnose the root cause, then open a pull request proposing the fix

You enable tracing and route OpenTelemetry spans through the LangSmith bridge with environment variables:

bash

export LANGSMITH_TRACING=true
export LANGSMITH_TRACING_MODE=otel

Cloud, hybrid or self-hosted?

There are three deployment models. The right one depends on your data-residency requirements. Cloud is the fully managed SaaS offering, with data residency in the US or EU, and suits teams that want to move fast. Hybrid keeps the control plane on SaaS for the UI while the data plane — where traces live — stays on your infrastructure. Self-hosted runs the entire stack inside your own VPC via Kubernetes or Terraform, which is what enterprises with strict residency rules need.

Three LangSmith deployment models — Cloud, Hybrid, and Self-hosted — distinguished by where the control plane and the data plane sit

A self-hosted instance runs several core services:

Frontend: Nginx serving the UI and routing API requests.
Backend: CRUD operations and business logic.
Platform backend: run ingestion and high-volume authentication.
Playground: forwards requests to LLM providers for testing.
ACE backend: executes arbitrary code in an isolated environment.

The storage stack needs ClickHouse for high-volume trace and feedback data, PostgreSQL for operational data, and Redis or Valkey for queuing and caching, with blob storage (S3, GCS, or Azure Blob) recommended for large trace artifacts. Self-hosting is an Enterprise-tier add-on, typically on an annual contract.

Trace-based pricing and the costs that surprise teams

As of mid-2026 there are three tiers: Developer (free, 1 seat, 5k base traces per month), Plus ($39 per seat per month, 10k base traces included), and Enterprise (custom). Past the included allowance, base traces with 14-day retention cost $2.50 per 1k. Usage is also metered in LangChain Storage Units (LSU, $1.00) and LangChain Compute Units (LCU, $1.50).

The first one to watch is extended retention. Keeping traces longer than 14 days raises the price to $5.00 per 1k traces for 400-day retention. If you upgrade a base trace to extended after ingestion, the platform charges an additional $2.50 per 1k — which matters when you want to build a long-term evaluation dataset out of production data you already have.

The bigger trap is instrumentation granularity, because trace volume follows how you measure rather than how much traffic you serve. Take a support agent handling 50,000 sessions a month. Instrument the whole session as one execution and you emit 50,000 traces, roughly $100 in overage. Start a new execution per conversation turn — say six turns per session — and you emit 300,000 traces, roughly $725 in overage, for exactly the same user value. That 6x difference is an engineering decision, not growth.

The instrumentation granularity trap: the same 50,000 sessions billed per session versus per conversation turn produces a sixfold difference in trace count

Evaluation runs count as billable traces too, so configure sampling if you run judges at scale.

How LangSmith differs from Langfuse

The main alternative to LangSmith is Langfuse. Both are moving toward the OpenTelemetry GenAI conventions, but they differ in business model and in how deep the framework integration goes.

Criterion	LangSmith	Langfuse
License	Proprietary	MIT core
Billing unit	Trace	Unit (per observation)
Self-hosting	Enterprise add-on	Free, via Docker Compose
LangChain integration	Deepest, default	Good, via OTel or integration

The billing unit is the difference most teams miss. LangSmith bills per trace, and a trace can hold up to 25,000 steps; Langfuse bills per unit, so every span and every score counts separately. For chatty agents making many tool calls per turn, LangSmith's trace-based model is more predictable, while Langfuse's per-unit model scales linearly with internal complexity.

On positioning, LangSmith is the bundled platform: a managed runtime, an Engine that proposes fixes, all running on SmithDB. Langfuse is the open-source option for teams that want full ownership of the data plane without an enterprise contract — and it kept its open-source commitment after ClickHouse acquired it in January 2026.

When is LangSmith the right choice?

Choose LangSmith when your system is already built on LangChain or LangGraph: the integration is close to zero-config, and you get visibility into cyclic agent loops that general-purpose tools struggle to reconstruct. The Engine adds to that if your team would rather have production failure modes surfaced automatically than read traces by hand.

If your volume is high and your budget is tight, price it carefully first — trace costs climb fast under poorly designed instrumentation, and that bill arrives long before you need the managed runtime. The decision comes down to ecosystem depth (LangSmith) against data-plane ownership (Langfuse); pick based on which one you are not willing to give up.