Evaluation tooling has a peculiar adoption pattern. Teams try it, like it, run evals for two weeks, and then quietly stop, because evaluating turned out to require a push step that nobody maintained.
So this comparison weights three things that predict whether a platform is still in use in month six: whether self-hosting is available without a sales call, how cost scales as evaluation becomes a habit rather than an experiment, and how much work it takes to get data in.
Licences, tiers and prices below were checked against each project's own repository and pricing page in August 2026.
Braintrust
The best pure evaluation product here, and worth saying first.
Scorer authoring, experiment workflow and review tooling are more polished than anything else on this list. If evaluation is your day job rather than one of six things you own, this is the tool that will feel best to use.
Pricing: Starter is $0 with $10 in monthly credits, 1 GB of processed data and 10,000 scores. Pro is $249 per month with $249 in credits, 5 GB and 50,000 scores, plus custom charts and priority support. Enterprise is custom.
The thing most roundups leave out: the core is proprietary. There is no public repository for the Braintrust platform. Their GitHub org publishes SDKs, tools and examples, and their own self-hosting docs state plainly that the deployed architecture is not open source. Two satellite libraries genuinely are MIT and usable on their own without an account, autoevals (scoring functions) and braintrust-proxy (a multi-provider API proxy), but the platform itself is closed.
That is a legitimate business model and Braintrust is not hiding it. It just means Braintrust is not competing on the axis the rest of this post is about, and a reader comparing licence terms should know that before comparing anything else.
Self-hosting is hybrid, not self-contained. The control plane, meaning the web UI, auth and metadata, stays hosted by Braintrust. Only the data plane runs in your cloud: Postgres, Redis, object storage and Brainstore, their proprietary trace-ingestion engine, deployed via Terraform modules they author. For privacy-sensitive teams that is a genuinely good split, since traces, prompts and scores never leave your infrastructure. For anyone whose requirement is "no vendor in the path at all", it does not meet the bar.
The two other things to weigh:
Self-hosting and hybrid deployment are Enterprise only. On-prem or hosted deployment for high volume or privacy-sensitive data is described as a custom Enterprise arrangement, not available on Starter or Pro. If a deployment constraint is why you are shopping, that changes the comparison immediately.
Pricing meters processed data and scores. That is fair and legible, and it also means cost rises exactly as evaluation becomes routine. Model your intended volume, not your pilot volume.
Pick it when: evaluation depth is the priority and hosted is acceptable.
Langfuse
MIT core, plus an enterprise licence over ee/. 33,900+ stars. ClickHouse-owned since January 2026.
Evaluations sit alongside tracing, datasets and prompt management in one product, which is a genuinely useful combination: the trace you want to score is already there.
Core tracing, evaluations, datasets, prompt management and the playground are free to self-host without usage limits, and that is a strong free offering. So are LLM-as-judge evaluators, human annotation queues, enterprise SSO and organisation-level RBAC. There is no seat cap or evaluator cap on the free build.
What needs a licence key when self-hosting: project-level RBAC, protected prompt labels, data retention policies, audit logs, server-side data masking, UI customisation, organisation creators, the Org Management API and SCIM, and the Instance Management API.
Read that list with a compliance hat on. If you are self-hosting evaluations because the data is sensitive, then retention policies and server-side data masking are precisely the features you need, and they are on the paid side.
Cloud pricing: Hobby free with 50,000 units per month, Core $29, Pro $199, Enterprise $2,499 per month.
Pick it when: you want tracing and evaluation in one tool and the gated list does not include what you need.
Opik
Apache-2.0, no ee/ directory, 21,697 stars, from Comet ML.
The largest community of the genuinely-clean-licence options, and actively developed. Self-hosting gives you all features including tracing and evaluation, minus user management. So you lose RBAC and multi-user administration, not evaluation capability. That is a meaningfully better deal than gating retention or masking.
The cost is operational. Its deployment compose file runs MySQL, Redis, ClickHouse, Zookeeper and MinIO. Five datastores. If your platform team already runs ClickHouse this is a shrug; if you are three engineers it is a weekly tax.
Pricing: Open Source free and self-hosted with the full feature set, Free Cloud at $0 with 10 members and 25,000 spans per month, Pro Cloud at $19 per month, Enterprise custom with SOC 2, ISO 27001, HIPAA and GDPR.
Pick it when: you want a complete self-hosted build with a clean licence and can absorb the operational weight.
OpenLIT
Apache-2.0, one datastore, no paid tier.
The lightest thing here by a distance: SDK to OpenTelemetry Collector to ClickHouse, and that is the whole architecture. Evaluations are one part of a broader Apache-2.0 project that also covers LLM observability, GPU monitoring, guardrails, prompt management and a secrets vault.
The honest weakness: there is no managed cloud. "OpenLIT Cloud" is listed as coming soon with no pricing. If you need a hosted product with an SLA, this project alone has nowhere for you to go, and planning around a "coming soon" is not a strategy.
Evaluation depth is also shallower than Braintrust's. It is a capable component of a broad platform rather than a specialist tool.
Pick it when: you want maximum capability per unit of operational burden and will self-host permanently.
Everstack
We build this, so judge the specifics.
Datasets, scorers, LLM judges, human annotation queues and regression analysis, self-hosted by default at any tier under one Apache-2.0 licence with no enterprise directory.
The structural difference is where the runs come from. Every other platform here evaluates data you instrument and send, because none of them are in the request path. Everstack scores runs that already went through the gateway or agent runtime, so there is no push step to maintain.
That matters because the push step is the thing that decays. Eval coverage goes partial when somebody ships a new service and nobody adds instrumentation to it.
Do not pick it for evaluation depth. Braintrust's eval product is better than ours, and if evals are the centre of your work you should use theirs. The case for Everstack is when evaluation is one of several problems and you want them in one place.
The three questions
1. Is self-hosting available without a sales call?
| Platform | Self-hosting | | --- | --- | | Braintrust | Enterprise only | | Langfuse | Yes, minus nine gated features | | Opik | Yes, minus user management | | OpenLIT | Yes, the only distribution model | | Everstack | Yes, at any tier |
2. How does cost scale as evaluation becomes routine? Per-GB and per-score pricing is cheap during a pilot and gets expensive exactly when the habit sticks. That is not a criticism of the model, it is a reason to model real volume before adopting.
3. How does data get in? Instrumentation is a code change in every service you want covered, and it is the step that decays. If your model calls already route through something, scoring what it recorded is strictly less work than pushing data to a second system.
Beyond that, the honest answer is that these are all decent products. Braintrust is the best at evaluation specifically, OpenLIT is the lightest to run, Opik has the best combination of clean licence and community size, and Langfuse has the most complete surrounding product. Pick on the three questions above rather than on a feature grid, because the feature grids converge and the deployment terms do not.
Everstack's evaluations are documented here, with a detailed Braintrust comparison.