open-source agent eval tools

A Library, an SDK or a Platform

The license tells you what you may do with the code. It does not tell you how much of the product the code is. Before comparing licenses, sort each project into one of three kinds:

• Library. Metrics and a runner you call from your own code or CI. Scores land wherever you put them. DeepEval, Ragas, Promptfoo and autoevals are libraries.
• SDK for a hosted platform. The client is open; the storage, UI and judges it reports to are not. judgeval is Apache-2.0, and the Judgment Labs platform behind it is hosted, with self-hosting listed as coming soon.
• Platform. UI, API, workers and databases, all in the repository, deployable on your own machines. Langfuse, Latitude, Future AGI and Phoenix are platforms.

The Projects, and What Is Open in Each

Reading the License Before You Build on It

MIT and Apache-2.0 are permissive: use, modify and run the code, commercially, with attribution. Three wrinkles are worth checking before you commit a team to a project.

• Open core. Langfuse keeps enterprise features in ee folders under a separate license; project-level RBAC, audit logs, data retention management and SCIM need a key when self-hosted. Future AGI ships a LICENSE-EE file beside its Apache-2.0 license. The free part is large in both, but read which folder the feature you need lives in.
• Source-available is not open source. The Elastic License 2.0 on Phoenix says you "may not provide the software to third parties as a hosted or managed service". Running it for your own team is fine; reselling it as a service is not.
• License changes. Latitude moved from LGPL-3.0 to MIT on 2026-05-20. Changes can go the other way too, so pin a version and keep the license file of the release you deploy.

One operational detail belongs here as well: Future AGI's self-hosted install sends telemetry by default, and FUTURE_AGI_TELEMETRY_DISABLED=1 turns it off.

A Library Is Not a Monitoring System

A library gives you the judge call and the metric. It does not give you somewhere to keep ten million traces, a way to read a transcript next to its score, a schedule that scores production as it arrives, retention, access control or an alert when a pass rate drops. For CI that does not matter: the test run is the unit, and a red build is the alert. For production it is most of the work.

So most teams combine the two kinds: a library in CI, and a platform - open source or hosted - for production. The scoring logic can be shared between them, and some platforms are built to run it for you. Failproof AI runs your existing evaluation suite in the cloud without requiring a rewrite and reports each result as a score, metric or assertion beside the trace. That keeps your evaluation logic portable even when the platform around it is managed.

Why Self-Hosting an Eval Platform Is Hard

Running an evaluator library is easy. Operating a production evaluation platform means ingesting traces continuously, scheduling evaluations, managing judge credentials and retries, storing high-volume event data, controlling access, setting retention rules, upgrading databases and making sure alerts still fire when part of the stack fails.

• Infrastructure: Postgres may be enough for a small install. Higher-volume platforms add ClickHouse, Redis, object storage, queues and workers that each need backups, upgrades and monitoring.
• Evaluation operations: production scoring needs sampling, concurrency limits, retry rules, model-provider fallbacks and spend controls. A pytest suite does not supply those pieces.
• Failure analysis: scores tell you which sessions failed. Grouping related failures, finding patterns nobody wrote an evaluator for and sending useful alerts requires another layer.
• Security: SSO, RBAC, audit logs, retention controls and data masking are commonly kept in paid enterprise editions even when the core platform is open source.
• Maintenance: check release cadence, migration notes and issues affecting the exact databases and integrations you plan to run.

Self-hosting is still the right choice when data control or infrastructure ownership is a hard requirement. Price the engineering time alongside the license fee.

Using Open-Source Evals with Failproof AI

Failproof AI is not an open-source evaluation platform. Its MIT-licensed CLI runs local behavioral policies across supported agent harnesses, but it does not run evaluations. Failproof AI Cloud handles trace storage, querying, code-based and LLM-based evaluations, automated failure analysis, findings and alerts.

That split suits teams that want to keep evaluators in open code without operating the platform around them. You can bring your existing evaluation suite to the cloud, find recurring failures across production sessions, and turn a suitable tested fix into a policy for the CLI to enforce locally.

Which to Start With

• Unit tests for an agent, in CI: DeepEval or Promptfoo.
• A RAG pipeline: Ragas, which is built around retrieval and answer quality.
• A UI and production traces on your own servers, free: Langfuse or Latitude.
• The broadest single open-source platform, gateway included: Future AGI.
• Keeping your evaluator code without running the surrounding platform: Failproof AI Cloud runs your existing evaluation suite on production sessions.

If the whole evaluation platform must be open source, choose Langfuse, Latitude or another self-hostable platform from the table. Failproof AI's open-source component is the local policy runtime, while its evaluation and failure-intelligence platform is hosted.

When Open Source Does Not Matter

Open source matters when you must self-host, want to read what touches your data, or need an exit that does not depend on a vendor. If none of those apply, a hosted tier with a clean export is often the better buy: someone else runs the database. Check what you can export before you sign, and keep your judges as your own code so they move with you. Migrating between platforms covers what moves and what does not.