Self-hosting guide
Self-hosted LLM observability: when the cloud is off the table
Self-hosted LLM observability for teams that need data residency, cost control, or OSS-only. Compare Langfuse, Phoenix, Helicone, OpenLLMetry, and DIY.
What to look for
Before you compare products, decide your acceptance criteria. Tracing fidelity — can it model nested LLM calls, tool I/O, and intermediate reasoning? Eval support — heuristic, LLM-as-judge, and human review. OpenTelemetry compatibility — can you export to an existing observability backend if you want. Deployment shape — single binary versus a cluster. Database dependency — Postgres, ClickHouse, or none. License — permissive OSS or source-available. Release cadence — weekly, monthly, ad hoc. These are the levers that matter; everything else is polish.
The realistic shortlist
Four options cover roughly 90% of teams that self-host.
Langfuse is MIT-licensed, ships a Docker compose and a Helm chart, depends on Postgres, has excellent DX, and moves fast — a meaningful release every few weeks. It is the most product-polished of the OSS options.
Arize Phoenix is Apache 2.0 and OpenTelemetry-native. If your platform team already standardises on OTel, Phoenix slots in without teaching anyone a new schema. It is lighter on the dashboard side and stronger on evals, especially for teams that already have an eval philosophy and want raw OTel spans.
Helicone is MIT-licensed and focused on cost and caching rather than the full observability surface. It is a strong fit if your main pain is token spend and you want a proxy you can drop in front of your LLM provider. Tracing is present but narrower than Langfuse or Phoenix.
OpenLLMetry plus your own backend is the DIY path: instrument your code with OpenLLMetry, then export traces to your existing observability stack — Tempo, Jaeger, Honeycomb, Datadog APM. The upside is zero new infrastructure for traces. The downside is that you get none of the LLM-specific affordances — prompt versioning, dataset management, eval UIs — out of the box.
What you give up
Self-hosting is not free. You give up managed upgrades, vendor support SLAs, automatic scaling, and — depending on the tool — the polished UI. Budget roughly half an FTE to operate the platform: upgrades, backups, capacity planning, and the inevitable incident at 2am. For a small team, that is real. For a team that already runs Kubernetes, it is incremental.
Pick by reason
If your driver is data residency, Phoenix or Langfuse on your own VPC is the obvious answer. If your driver is cost at scale, Helicone (because the caching wins compound) or Langfuse (because its usage-based pricing is predictable) is the right call. If your driver is OSS-only procurement, Langfuse or Phoenix both qualify under MIT and Apache 2.0 respectively. If you already have a serious observability stack and you only need LLM-aware instrumentation, OpenLLMetry into your existing backend is the lowest-friction path.
Frequently asked questions
How much engineering does self-hosted Langfuse take?
Most teams get a working deploy in a day using the official Docker compose. Operating it is two to four hours per month at low volume, rising with the number of environments and the size of the trace backlog. Plan an upgrade cadence of every two to four weeks to stay current with security patches.
Can I mix self-hosted traces with a cloud analytics tool?
Yes. A common split is Plausible or a self-hosted Umami for product analytics on the public site, plus your LLM traces staying entirely inside your VPC. The two never share a data path, which is exactly what compliance teams usually want.
Is there a fully managed open-source option?
Not exactly. The closest thing is the cloud version of Langfuse or Arize, which run the same OSS code on their infrastructure. You get managed operations without giving up the ability to fork and self-host later.