# What is LangSmith? Observability and Evals for LangChain Agents

> LangSmith is a framework-agnostic platform for agent observability, automated evaluation, and managed deployment across the development lifecycle.

Source: https://camnangai.com/en/engineering/what-is-langsmith  
Published: 2026-08-15

[LangSmith](https://www.langchain.com/langsmith-platform) is a framework-agnostic agent
  engineering platform built around three pillars: observability, evaluation, and deployment.

You use it to handle the non-deterministic behavior of [large language
models](https://camnangai.com/en/knowledge/what-is-an-llm) (LLMs) by tracing and inspecting every intermediate step in
the system's reasoning path.

The platform lets you see inside the black box of an [AI agent](https://camnangai.com/en/knowledge/what-is-an-ai-agent),
from early prompt experiments to production monitoring.


  Instead of guessing why an agent went wrong, you get structured data on [retrieval-augmented
  generation (RAG)](https://camnangai.com/en/knowledge/what-is-rag) flows, tool calls, and reasoning logic — enough to
  tune performance and keep costs under control.


![Seeing inside the black box of a running AI agent, with each reasoning step surfaced as observable data](https://camnangai.com/articles/langsmith-la-gi/01-infographic-en.webp)

## What is LangSmith?

**LangSmith** is a framework-agnostic platform. It integrates most deeply with the
[LangChain](https://camnangai.com/en/engineering/what-is-langchain) ecosystem, but you can instrument custom agent
systems through the [Python](https://camnangai.com/en/engineering/what-is-python) and JS/TS SDKs, or through
OpenTelemetry. It is the control layer for an agent's whole development lifecycle, not a logging
library.

![The agent development lifecycle in LangSmith — build, test, deploy, monitor, improve — alongside the three pillars of observability, evaluation, and deployment](https://camnangai.com/articles/langsmith-la-gi/02-en.webp)

Across that lifecycle, LangSmith shows up at every stage: the Playground for testing prompts and
seeing results immediately, tracing for recording complex execution paths, datasets for building
test suites out of real traffic, and monitoring once the system is in production. You can break
apart each node in a [LangGraph](https://camnangai.com/en/engineering/what-is-langgraph) graph, or each execution step
of a Deep Agent.

For a systems engineer, the real value is being able to measure what changes when you swap a model
or rewrite the instructions in a prompt. You get concrete numbers on latency, cost, and output
quality to show the new version beats the old one, rather than relying on trial and error.

## Traces, runs, threads and projects: what LangSmith records

To keep observability data manageable, **LangSmith** organizes it in a hierarchy. A **run** is the
atomic unit of work (a single model call, tool invocation, or vector DB retrieval), and it maps to a
span in OpenTelemetry terms. A **trace** is a tree of runs covering one user-facing operation. Each
trace is capped at 25,000 runs; past that, the system stops recording. 25,000 sounds generous until a long agent loop runs into it, and what you lose is the tail, which is usually the part you opened the trace to read.

![The nested shape of observability data: many runs inside one trace, many traces linked into a thread, with a trajectory as the flat view of the same data](https://camnangai.com/articles/langsmith-la-gi/03-en.webp)

A **thread** is a sequence of traces linked by a `thread_id`, representing one multi-turn session.
Where a thread preserves the nested structure, a **trajectory** gives you a flat, time-ordered list
of AI, Human, and Tool messages. Trajectories are faster to read because they strip the noise of
intermediate execution steps; threads keep the full technical detail you need for debugging.

All of this lives inside **projects**, which are separate containers per service or per environment
(dev, staging, prod).

| Concept    | Shape              | What it holds                              | What you use it for                      |
| :--------- | :----------------- | :----------------------------------------- | :--------------------------------------- |
| Run        | Single unit        | Input, output, metadata for one step       | Inspecting one specific operation        |
| Trace      | Tree of runs       | Every step within one operation            | Debugging logic failures or latency      |
| Thread     | Sequence of traces | Multi-turn conversation history            | Watching agent behavior across a session |
| Trajectory | Flat message list  | AI, Human, and Tool messages in time order | Reading the exchange quickly, less noise |

You enrich this data with feedback, tags, and metadata. Feedback attaches scores to individual
runs, which is the basis for both human review and automated evaluation. Tags are strings for high-level
filtering, and metadata holds key-value context such as application version or environment.

## Evaluating quality: datasets, experiments and LLM-as-judge

Evaluation in **LangSmith** splits into offline and online work. **Offline evaluation** happens
during development: you benchmark model versions and run regression tests against **datasets**,
which are collections of examples pairing an input with a reference output. Running your
application against a dataset produces an **experiment**, capturing outputs and scores so you can
compare configurations side by side.

![The evaluation loop: a failing production trace is added to a dataset, an experiment runs against it, then the old and new versions are compared](https://camnangai.com/articles/langsmith-la-gi/04-en.webp)

**Online evaluation** covers production monitoring. You have no reference outputs for live traffic,
so you lean on heuristics, safety checks, and reference-free LLM-as-judge evaluators. They watch
runs and threads in real time and flag quality degradation before it reaches users at scale. The
loop closes when an issue caught online becomes a new example in your offline dataset.

The LLM-as-judge pattern uses one model to grade another's output against a rubric encoded in a
prompt:

- **Reference-free evaluators** assess clarity, tone, or safety-policy adherence with no expected
  answer.
- **Reference-based evaluators** compare output against a ground-truth reference to check factual
  accuracy.

For subjective work like summarization, **pairwise evaluation**, where a human or LLM judge picks
the better of two outputs, often beats assigning absolute scores. To scale human review,
**annotation queues** give reviewers a structured workflow for scoring traces against a rubric. The catch with judges is that they are models too, so they inherit the blind spots you are trying to measure; they scale grading up, but they do not replace reading the output yourself.

## Context Hub: managing prompts and context like code

Instructions, policies, and examples change far faster than application code. **Context Hub** gives
you a central place to store and version the non-code parts of your agents: individual prompts
(message templates) and contexts (bundles of instructions, tools, and policies). Decoupling them
from your core logic lets domain experts change behavior without a code deploy.

![Context Hub tags each context version dev, staging, or prod, so changing an agent's behavior means moving a tag rather than redeploying code](https://camnangai.com/articles/langsmith-la-gi/05-en.webp)

It behaves like version control for agent logic. You tag versions `dev`, `staging`, or `prod` so
each agent pulls the right configuration for its environment, and teammates can comment on changes
and review history before promoting a version. That matters most in multi-agent systems where
several agents share a set of skills or instructions.

Context Hub is also where the files that define agent behavior live: `AGENTS.md` files, skill
sets, and project-specific examples. It connects to the **LangSmith Playground**, so you can test a
prompt or context bundle against different models before committing it. Context gets the same
governance and auditability you expect from source code. The trade-off is that agent behavior now lives outside your repository, so the version history in Context Hub has to be taken as seriously as your commit log.

## LangSmith Engine and SmithDB: what changed after Interrupt 2026

At Interrupt 2026, LangChain shipped two upgrades aimed at the scale of modern agent workloads.

**SmithDB** is a purpose-built database for agent observability, written in Rust on top of Apache
DataFusion and Vortex. It stores durable trace data in object storage and keeps ingestion and query
services stateless, which is what makes deeply nested spans and long-running agent operations
workable:

- Full trace tree load: 92ms (P50).
- Single run load: 71ms (P50).
- Up to 15x faster than the previous backend, while keeping the system stateless and easy to scale.

**LangSmith Engine** is the autonomous layer on top. It watches production data, clusters failures
into prioritized issues, and diagnoses root causes against your code and prompts. Once enabled, it
can propose fixes by opening pull requests or by creating online evaluators that catch the same
regression next time. Your role moves from manual debugging to reviewing and merging suggestions. That is a real shift, though it moves the burden rather than removing it, because a wrong pull request still costs a full read.

![The LangSmith Engine loop: watch production traces, cluster failures, diagnose the root cause, then open a pull request proposing the fix](https://camnangai.com/articles/langsmith-la-gi/06-en.webp)

You enable tracing and route OpenTelemetry spans through the LangSmith bridge with environment
variables:

```bash
export LANGSMITH_TRACING=true
export LANGSMITH_TRACING_MODE=otel
```

## Cloud, hybrid or self-hosted?

There are three deployment models. The right one depends on your data-residency requirements.
**Cloud** is the fully managed SaaS offering, with data residency in the US or EU, and suits teams
that want to move fast. **Hybrid** keeps the control plane on SaaS for the UI while the data plane
— where traces live — stays on your infrastructure. **Self-hosted** runs the entire stack inside
your own VPC via Kubernetes or Terraform, which is what enterprises with strict residency rules
need.

![Three LangSmith deployment models — Cloud, Hybrid, and Self-hosted — distinguished by where the control plane and the data plane sit](https://camnangai.com/articles/langsmith-la-gi/07-en.webp)

A self-hosted instance runs several core services:

- **Frontend:** Nginx serving the UI and routing API requests.
- **Backend:** CRUD operations and business logic.
- **Platform backend:** run ingestion and high-volume authentication.
- **Playground:** forwards requests to LLM providers for testing.
- **ACE backend:** executes arbitrary code in an isolated environment.

The storage stack needs **ClickHouse** for high-volume trace and feedback data, **PostgreSQL** for
operational data, and **Redis** or **Valkey** for queuing and caching, with **blob storage** (S3,
GCS, or Azure Blob) recommended for large trace artifacts. Self-hosting is an Enterprise-tier
add-on, typically on an annual contract.

## Trace-based pricing and the costs that surprise teams

As of mid-2026 there are three tiers: **Developer** (free, 1 seat, 5k base traces per month),
**Plus** ($39 per seat per month, 10k base traces included), and **Enterprise** (custom). Past the
included allowance, base traces with 14-day retention cost **$2.50 per 1k**. Usage is also metered
in LangChain Storage Units (LSU, $1.00) and LangChain Compute Units (LCU, $1.50).

The first one to watch is extended retention. Keeping traces longer than 14 days raises the price
to **$5.00 per 1k traces** for 400-day retention. If you upgrade a base trace to extended _after_
ingestion, the platform charges an additional **$2.50 per 1k** — which matters when you want to
build a long-term evaluation dataset out of production data you already have.

The bigger trap is instrumentation granularity, because trace volume follows how you measure rather
than how much traffic you serve. Take a support agent handling 50,000 sessions a month. Instrument
the whole session as one execution and you emit 50,000 traces, roughly $100 in overage. Start a new
execution per conversation turn — say six turns per session — and you emit 300,000 traces, roughly
$725 in overage, for exactly the same user value. That 6x difference is an engineering decision,
not growth. I would settle trace boundaries during design review, because working them out from an invoice means you have already paid for the wrong answer.

![The instrumentation granularity trap: the same 50,000 sessions billed per session versus per conversation turn produces a sixfold difference in trace count](https://camnangai.com/articles/langsmith-la-gi/08-en.webp)

Evaluation runs count as billable traces too, so configure sampling if you run judges at scale.

## How LangSmith differs from Langfuse

The main alternative to **LangSmith** is [Langfuse](https://camnangai.com/en/engineering/what-is-langfuse). Both are
moving toward the OpenTelemetry GenAI conventions, but they differ in business model and in how
deep the framework integration goes.

| Criterion             | LangSmith         | Langfuse                      |
| :-------------------- | :---------------- | :---------------------------- |
| License               | Proprietary       | MIT core                      |
| Billing unit          | Trace             | Unit (per observation)        |
| Self-hosting          | Enterprise add-on | Free, via Docker Compose      |
| LangChain integration | Deepest, default  | Good, via OTel or integration |

The billing unit is the difference most teams miss. LangSmith bills per trace, and a trace can hold
up to 25,000 steps; Langfuse bills per unit, so every span and every score counts separately. For
chatty agents making many tool calls per turn, LangSmith's trace-based model is more predictable,
while Langfuse's per-unit model scales linearly with internal complexity.

On positioning, LangSmith is the bundled platform: a managed runtime, an Engine that proposes
fixes, all running on SmithDB. Langfuse is the open-source option for teams that want full
ownership of the data plane without an enterprise contract — and it kept its open-source commitment
after ClickHouse acquired it in January 2026.

## When is LangSmith the right choice?

Choose **LangSmith** when your system is already built on LangChain or LangGraph: the integration
is close to zero-config, and you get visibility into cyclic agent loops that general-purpose tools
struggle to reconstruct. The Engine adds to that if your team would rather have production failure
modes surfaced automatically than read traces by hand.

If your volume is high and your budget is tight, price it carefully first — trace costs climb fast
under poorly designed instrumentation, and that bill arrives long before you need the managed
runtime. The decision comes down to ecosystem depth (LangSmith) against data-plane ownership
([Langfuse](https://camnangai.com/en/engineering/what-is-langfuse)); pick based on which one you are not willing to give
up.

## References

- [LangSmith: AI Agent & LLM Observability and Evals Platform — LangChain](https://www.langchain.com/langsmith-platform)
- [Observability concepts — Docs by LangChain](https://docs.langchain.com/langsmith/observability-concepts)
- [Evaluation concepts — Docs by LangChain](https://docs.langchain.com/langsmith/evaluation-concepts)
- [Prompt & Context Hub — Docs by LangChain](https://docs.langchain.com/langsmith/context-hub)
- [Self-hosted LangSmith — Docs by LangChain](https://docs.langchain.com/langsmith/self-hosted)
- [LangSmith Plans and Pricing — LangChain](https://www.langchain.com/pricing)
- [Everything we shipped at Interrupt — LangChain](https://www.langchain.com/blog/interrupt-2026-overview)
- [Top LLM Observability and Evaluation Platforms in 2026: Langfuse, LangSmith, Braintrust, Arize, and More Compared — MarkTechPost](https://www.marktechpost.com/2026/08/09/top-llm-observability-and-evaluation-platforms-in-2026-langfuse-langsmith-braintrust-arize-and-more-compared/)
- [LangSmith Pricing Explained (2026): Traces, Seats & Alternatives — Inference.net](https://inference.net/content/langsmith-pricing/)