LLM Observability and Evaluation Tools With a Free Tier (2026)

If you ship an AI feature in a web app and want to see what it does in production, Langfuse, Opik and Arize Phoenix give you free tracing either hosted or self-hosted, and all three hook into the Vercel AI SDK. If you want evals that run in CI before a prompt change merges, promptfoo (Node) and DeepEval (Python) are open source and free to run locally. LangSmith, Braintrust and Helicone have hosted free plans with tighter caps.

Every limit, license and package name below was checked against each vendor's own pricing page, docs or GitHub README on 11 November 2026. Where a vendor does not publish a figure, the entry says so.

Tracing vs evals, briefly

Tracing records what happened on each real request: the prompt, the model's answer, tool calls, tokens, latency and cost, nested as spans inside one trace. You read traces to debug a bad answer a user got, or to see where the money goes. It looks backwards, at live traffic.

Evals score outputs against expectations: a dataset of inputs, a check per case (exact match, a regex, or an LLM acting as judge), and a pass rate you compare across prompt or model versions. You run evals before shipping a change, often in CI. It looks forward, at a change you are about to make.

Most hosted platforms now do both, and the useful ones connect them: a bad trace becomes a dataset row, and the dataset becomes the eval. The test-runner tools here (promptfoo, DeepEval) do evals only.

Platforms: tracing and evals

Langfuse

Langfuse Cloud's free Hobby plan includes 50k units a month, 30 days of data access and 2 users, with no credit card; extra usage is not offered on Hobby. The repository is MIT licensed except the ee folders, and the README describes a single-VM setup with docker compose up. A production self-host needs Postgres, Redis or Valkey, ClickHouse and S3-compatible storage; "some add-on features require a license key".

Langfuse is built on OpenTelemetry and has typed SDKs for Python and JS/TS. The Vercel AI SDK integration exports the AI SDK's spans through LangfuseSpanProcessor from @langfuse/otel; on AI SDK 7 you also register @langfuse/vercel-ai-sdk. In serverless functions, call forceFlush() so traces leave before the function stops.

LangSmith

LangSmith's free Developer plan is one seat and 5k base traces a month, then pay-as-you-go; base traces are kept for 14 days. Self-hosting is only on Enterprise. It covers both halves: offline evals on datasets and online evals on live traffic, per its evaluation docs.

For TypeScript, the langsmith npm package traces the Vercel AI SDK: wrapAISDK on AI SDK v5 and v6, or registerTelemetry(LangSmithTelemetry()) on v7. That page treats the OpenTelemetry route as a legacy guide.

Braintrust

Braintrust's free Starter plan is metered in data, not traces: 1 GB processed, 10,000 scores and $10 of model credits, with 14 days of retention. Users, projects and experiments are unlimited. On-prem or hosted deployment is Enterprise only. Its docs cover both logging traces and running evals that compare prompts and models.

The TypeScript package is braintrust. For the Vercel AI SDK it offers wrapAISDK (v3 to v6), braintrustAISDKTelemetry() (v7), a Node import hook, a Next.js config wrapper, or plain OpenTelemetry through @braintrust/otel.

Opik (Comet)

Opik Free Cloud gives you 25,000 spans a month, 60 days of retention and up to 10 team members. The open-source version is the same codebase, and the repository is Apache-2.0, "free to self-host the full platform", started with ./opik.sh (Docker Compose) or a Helm chart.

It has SDKs for Python and TypeScript and accepts OpenTelemetry from any language. Evals come as datasets, experiments and LLM-as-a-judge metrics, with a PyTest integration for CI. The Vercel AI SDK integration is the opik-vercel package: an OpikExporter you pass to registerOTel in Next.js, for AI SDK v4 through v7.

Arize Phoenix

Phoenix is the self-hosted, local-first part of Arize: pip install arize-phoenix then phoenix serve, a Docker image or a Helm chart. Its license is the Elastic License 2.0, which is source-available rather than an OSI license. If you would rather not host it, the Arize AX Free plan gives 25,000 spans and 1 GB of ingestion a month, 15 days of retention and unlimited users.

Phoenix is built on OpenTelemetry, with instrumentation from the OpenInference project. On npm you get @arizeai/phoenix-otel, @arizeai/phoenix-client and @arizeai/phoenix-evals (still alpha), and @arizeai/openinference-vercel for the Vercel AI SDK.

Tracing first: Helicone

Helicone works as a gateway: you point your client at its endpoint and every request is logged on the way through. The free Hobby plan has 10,000 requests a month, 1 GB of storage, 7 days of retention and one seat. The repository is Apache-2.0 and self-hosts with Docker Compose; the pricing page lists on-prem only under Enterprise.

For the Vercel AI SDK there is a provider package, @helicone/ai-sdk-provider: you call helicone('model-name') instead of a provider's model. Its docs we checked describe sessions, caching, rate limits and prompt management, but not evals, so plan on another tool for those.

Evals in CI: promptfoo and DeepEval

promptfoo

promptfoo is an MIT-licensed CLI that runs evals and red-team scans locally: npx promptfoo@latest init to start, or install it with npm, Homebrew or pip. The README notes it "is now part of OpenAI" and stays open source. The Community version is free, with red teaming capped at 10k probes a month; Enterprise and on-prem tiers add sharing, monitoring and SSO.

The GitHub Action promptfoo/promptfoo-action@v1 runs a before-and-after comparison on every pull request that changes a prompt and posts the result as a PR comment. No tracing: this is a test runner.

DeepEval

DeepEval is an Apache-2.0 eval framework for Python 3.9+, installed with pip install -U deepeval. Tests read like pytest and run with deepeval test run, so they fit any CI. The repo has TypeScript examples for the AI SDK, but the framework itself is Python, which matters if your web app has no Python anywhere.

deepeval login sends results to Confident AI, its hosted platform. The free plan there is 2 seats, 1 project, 5 test runs a week and 1 GB-month of trace spans; retention is not published.

Side by side

Tool	Does	Free hosted tier	Retention	Seats	Self-host license
Langfuse	Both	50k units/month	30 days	2	MIT (except ee)
LangSmith	Both	5k base traces/month	14 days	1	Enterprise only
Braintrust	Both	1 GB processed, 10k scores	14 days	Unlimited	Enterprise only
Opik	Both	25k spans/month	60 days	Up to 10	Apache-2.0
Arize Phoenix / AX	Both	25k spans, 1 GB/month (AX)	15 days	Unlimited	ELv2
Helicone	Tracing	10k requests/month	7 days	1	Apache-2.0
promptfoo	Evals	Local CLI, free	Local	n/a	MIT
DeepEval	Evals	Confident AI: 5 runs/week	Not published	2	Apache-2.0

All six platforms have a TypeScript SDK or package for the Vercel AI SDK. Langfuse, Opik, Phoenix and Braintrust document an OpenTelemetry path for it; LangSmith ships its own wrapper and Helicone works as a gateway provider.

How to choose

• You just want traces. Start with Langfuse or Opik: both trace the Vercel AI SDK over OpenTelemetry and have among the largest free hosted caps here. If you would rather not touch OpenTelemetry at all, Helicone's AI SDK provider is the smallest change, at a lower cap and 7-day retention.
• You want evals in CI. promptfoo if your stack is Node: one config file and a GitHub Action that comments on the PR. DeepEval if you already write Python tests. If you also want the results next to production traces, Langfuse, Opik, LangSmith and Braintrust keep datasets and experiments in the same place.
• You must self-host. Opik (Apache-2.0, whole platform) and Langfuse (MIT core, some add-ons licensed) are the two full platforms you can run yourself. Phoenix is free to run but under ELv2. LangSmith and Braintrust self-host only on Enterprise.
• You are a team on a free plan. Seat counts differ more than trace caps: LangSmith and Helicone are one seat; Braintrust and Arize AX are unlimited.

Tracing also shows what each model call costs you, which is worth knowing before a free model tier runs out. The free AI API tiers post lists the providers you can point these tools at.

FAQ

What is the difference between LLM tracing and LLM evals?

Tracing records real requests as they happen (prompt, output, tool calls, tokens, latency) so you can debug and measure production. Evals run a fixed dataset through your app and score the outputs, so you can compare versions before you ship. Most platforms here do both.

Which LLM observability tools can I self-host for free?

Langfuse (MIT, except the ee folders), Opik (Apache-2.0), Helicone (Apache-2.0) and Arize Phoenix (Elastic License 2.0). promptfoo and DeepEval run locally as open-source tools. LangSmith and Braintrust offer self-hosting only on Enterprise plans.

Do these tools work with the Vercel AI SDK?

All six platforms document it. Langfuse, Opik, Phoenix and Braintrust export the AI SDK's OpenTelemetry spans; LangSmith and Braintrust also offer a wrapAISDK wrapper; Helicone ships @helicone/ai-sdk-provider. Check the AI SDK version each guide targets, since v7 changed how telemetry is registered.

Which free tier is the most generous?

By the vendors' own numbers: Langfuse's 50k units a month with 30-day access, and Opik's 25k spans with 60-day retention and up to 10 members. The units differ, so compare against how many spans one of your requests produces, not the headline figure.

---

For the models these tools watch, see the free AI API tiers that last. More AI tooling for web developers is in the AI & ML directory, and the rest of the developer tools are under Devtools.