judgment labs vs galileo

Two Evaluation Companies with Different Buyers

Judgment Labs is built around judges. The open-source judgeval SDK traces agent runs over OpenTelemetry; judges score those traces; Agent Behavior Monitoring runs the judges continuously and fires Slack alerts, webhooks and dataset actions. Its newer work is about what to score in the first place: Behaviors are semantic tags over production sessions, Behavior Discovery is described as "surfacing failure modes and usage patterns from unlabeled production trajectories", and AutoRubrics as "automatically constructing and refining evaluation rubrics from verifiable signals". The company raised $32M across seed and Series A, both led by Lightspeed, in May 2026.

Galileo is built around evaluation. It ships 20+ out-of-box evals for RAG, agents, safety and security, custom evaluators, the Luna-2 small evaluation models, and CLHF for tuning a judge from a handful of labels. Cisco announced its intent to acquire Galileo on April 9, 2026; Splunk said on May 28 that the acquisition is complete and that Galileo's technology will be built into Splunk Observability's AI Agent Monitoring. Galileo's release notes add that "as of August 7, 2026, Galileo is now Splunk Agent Observability"; this page keeps the Galileo name, which is still the one on galileo.ai.

Both are evaluation companies, but they are built for different rooms. Judgment Labs names its audience as "agent-native companies" and teams building deep agents, and the only call to action on its site is "Book demo". Galileo lists Twilio, Comcast, HP and ServiceTitan among its customers, lets you start on its Free tier without talking to anyone, and keeps its heaviest features behind an Enterprise contract; Pro and Enterprise both start from a "Book a Demo" button. Before comparing judges, check which room you are standing in.

Side by Side

The table is two different bets. Judgment Labs bets that the scarce thing is judgment itself: a rubric good enough to grade a forty-step trajectory, and a way to find the failure you did not know to write a rubric for. Galileo bets that the scarce thing is coverage and control: a metric for most common failures on day one, and a path from an offline eval to something that sits in front of production traffic. Which bet you want depends on whether your problem is "we do not know what is going wrong" or "we know, and we need it handled at scale".

How Each One Judges a Run

Judgment Labs has two judge types. An Agent Judge follows "a natural-language rubric and reasons over the supplied trace"; a Code Judge runs "Python logic that you implement". Its docs draw the line clearly: use the Agent Judge when the decision needs interpretation, the Code Judge when the condition is exact, such as a timeout, an exception or an empty result. Judges return a binary, categorical or numeric output with reasoning, and run online, offline against datasets, or on demand. One detail matters for monitoring: numeric judges can score a live trace when you apply them, but cannot be enabled as continuously monitored behaviors, so ongoing monitoring is framed as true or false, or a category.

Galileo starts from a library. Its 20+ out-of-box evals cover RAG, agents, safety and security; the Agentic Evaluations it launched in January 2025 include Tool Selection Quality, Tool Call Error Detection and Session Success. On top of that you write custom evaluators, and Galileo distills expensive LLM-as-judge evaluators into Luna-2, fine-tuned 3B and 8B models priced at $0.02 per million tokens. Galileo's docs say plainly that "Luna-2 is only available in the Enterprise tier". CLHF lets you correct a judge with human feedback from as few as five annotated records.

Where Judgment Labs Is Stronger

• Long trajectories. Agent Judge is built for trajectory-level grading, and the team published its long-context evaluation work in May 2026.
• The unknown unknowns. Behavior Discovery works on unlabeled production trajectories, which is where most failure modes live before anyone names them.
• Languages. judgeval has JavaScript, Go and Java siblings next to the Python library.

Where Galileo Is Stronger

• Day-one coverage. Twenty-odd metrics you do not have to write, including agent-specific ones for tool use and session outcome.
• Cost of judging at volume. Luna-2 exists to make judging every trace affordable, if you are on Enterprise.
• Human correction. CLHF turns a few reviewer labels into a better metric without rewriting a prompt.

Guardrails: Where Galileo Goes Further

Judgment Labs does not block anything, by design. Its Agent Behavior Monitoring docs describe judging the "completed traces" an instrumented agent sends, with measures covering "traces the judge evaluated, not necessarily every trace sent to the project", and automations that "notify a team or trigger an action". The judgeval README says it scores "live production traffic server-side with no latency impact". Keeping judges off the hot path means a slow judge never slows your agent, and it also means the alert arrives after the agent has acted.

Galileo has two ways to act at runtime. The hosted route is "Real-time guardrails", which the pricing page lists under Enterprise only; its docs pitch Luna-2, with a stated 152ms average latency, as what makes runtime protection practical. The open-source route is Agent Control, Apache-2.0, launched in March 2026: a decorator at your agent's decision points sends an LLM or tool input or output to the Agent Control server, which checks it against the active policy and returns deny, steer, warn, log or allow. You run it yourself, as a server backed by PostgreSQL, with Python and TypeScript SDKs and integrations for LangChain, CrewAI, Google ADK and AWS Strands; a centrally managed version for enterprise customers shipped in June 2026. The older Galileo Protect was deprecated in June 2026, and the docs tell new users to set up Agent Control instead.

If a runtime control is on your requirements list, only one of these two vendors has one. If it is not, Judgment Labs' after-the-fact design costs you nothing, and is simpler to put in front of a latency-sensitive agent.

One Security Review, Both Vendors

Take a 300-person insurer whose platform team has a short list from security: traces stay in our cloud account, SSO for everyone, a price before the purchase order, and a control that stops customer data from leaving in a tool call. Here is how each vendor meets the list as of September 2026.

1. Trying it this week

   Galileo: sign up for Free, which covers 5,000 traces a month with unlimited users and unlimited custom evals, and instrument one agent. Judgment Labs: judgeval installs from PyPI under Apache-2.0, but the platform it reports to starts with a demo booking.

2. Keeping traces in your own account

   Galileo offers hosted, VPC and on-prem deployment on Enterprise. Judgment Labs is hosted; its docs say "Self-hosting capabilities for the Judgment Platform are coming soon", and VPC or on-prem terms are not published.

3. SSO

   Galileo puts SSO on Enterprise; Pro includes standard RBAC. Judgment Labs does not publish which plan includes SSO, because it publishes no plans.

4. Stopping data leaving in a tool call

   Galileo: hosted real-time guardrails on Enterprise, or Agent Control, which your team deploys and operates. Judgment Labs: a judge can flag the session afterwards; stopping it needs a separate enforcement layer.

5. A price before the purchase order

   Galileo Pro is $100 a month billed yearly for 50,000 traces, and "pricing scales based on number of traces"; like Enterprise, Pro starts from a "Book a Demo" button, and everything security asked for is on Enterprise. Judgment Labs: not published, for any tier.

On paper Galileo clears more of the list. Read it again, though: almost every box it clears is an Enterprise box, which is also a sales conversation. The real difference is that with Galileo you know which tier you are negotiating for before the call, and with Judgment Labs the call is where you find out.

Pricing, Ownership and What to Ask

Galileo's ownership changed this year, and that deserves a question rather than a guess. The public facts are the Cisco announcement, the Splunk statement that the deal closed, and the plan to build Galileo's technology into Splunk Observability. Nothing public says how standalone plans, contracts or data processing will change. If you are signing a multi-year Enterprise agreement, ask which entity you contract with, whether the published tiers are committed for your term, and where your data is processed once integration work starts.

For Judgment Labs, the demo is where the pricing lives, so go in with a list: how price scales with trace volume and judge runs, what sampling rate monitoring uses by default, retention, which plan includes SSO, whether there is a data processing agreement, and a date for self-hosting. None of those answers is on the site today, and "not published" is not the same as "expensive". It is the same as "unknown until you ask".