Is judgeval open source?

What Is Open and What Is Not

Apache-2.0 is a permissive license. You can use, modify and redistribute the SDK, including inside commercial software, as long as you keep the license and notices, and it includes an express patent grant from contributors. For a client library, that is about as open as it gets.

What the Open SDK Gives You

Per its README, judgeval provides:

• Tracing built on OpenTelemetry, through a @Tracer.observe() decorator, with auto-instrumentation for OpenAI, Anthropic, Google GenAI and Together AI clients.
• Framework integrations for LangGraph, OpenLit and the Claude Agent SDK.
• Agent judges, described as "prompt-based scorers to evaluate agent behaviors at scale".
• Online monitoring: "Automatically score live production traffic server-side with no latency impact."
• JQL, a query interface, and an MCP server. The README also describes a CLI, but Judgment Labs' CLI reference marks it deprecated, so plan around the SDK and MCP.

Read the list for where the work happens. Tracing runs in your process, and that code is open: you can read exactly what the SDK captures and sends, patch it, or pin it. Scoring live traffic runs "server-side", on the Judgment platform, and so does storage. That is why the quickstart starts by exporting JUDGMENT_API_KEY and JUDGMENT_ORG_ID.

What an Open SDK Does Not Give You

An open client in front of a closed service is a common and reasonable model. It answers some questions and not others:

• It answers "what does this library send from our process?" You can audit it line by line, which helps a security review.
• It does not answer "where is our data stored?" Traces go to the hosted platform, and self-hosting that platform is not available yet. See can you self-host Judgment Labs.
• It does not make you portable by itself. Tracing on OpenTelemetry is the portable layer. Judges, rubrics, behaviors and monitoring configuration are platform features, so check what can be exported before you build on them.
• It does not set the price. The SDK is free; the platform's pricing is not published. See Judgment Labs pricing.

Getting the Most from the Open Part

If you adopt judgeval, the license gives you three practical levers. Each is worth pulling before you are deep in the platform:

• Audit the payload. Before production, read the instrumentation for the clients you use - OpenAI, Anthropic, Google GenAI, Together AI - and confirm what each span carries: prompts, completions, tool arguments, token counts. If a field must never leave your network, strip it in your own code before the traced call rather than trusting a setting you cannot see.
• Pin the version. judgeval needs Python 3.10 or later. Pin the current PyPI release like any dependency that sends data off the machine, and read the release notes before upgrading, because an instrumentation change can change what leaves your process.
• Keep the traces portable. Tracing is OpenTelemetry, so keep your own span names and attributes meaningful rather than leaning on vendor-specific decoration. If you later add or switch back ends, OpenTelemetry-shaped instrumentation is the part most likely to carry over.

None of this changes where hosted judging, monitoring and history live. If you want to operate the complete platform yourself, compare Latitude and Langfuse with the operational work each requires. If managed infrastructure is acceptable, Failproof AI runs code-based and LLM-based evaluations in the cloud, supports your existing evaluation suite, and analyzes trace evidence across sessions to find recurring failure modes and recommend fixes.

How the Deployment Models Compare

If "open source" means you can run the complete core platform yourself, Latitude and Langfuse are the clearest options here, though Langfuse keeps enterprise features in ee/. Future AGI publishes an Apache-2.0 core with separately licensed enterprise code. If the requirement is to audit the code running inside the agent, judgeval qualifies, as does the MIT-licensed Failproof AI CLI. Failproof AI Cloud remains managed or Enterprise self-hosted. Open-source agent eval tools covers the build and operating tradeoffs in more depth.

What the License Changes

The Apache-2.0 license lets you inspect what leaves your process, patch an integration and keep the instrumentation if you change back ends. It does not give you the hosted platform, its stored history or a self-hosted deployment. For a purchase decision, evaluate those separately: SDK license, data location, export format, operational ownership and platform price.