evals vs guardrails

What Each One Is

An eval is a measurement. It takes something the agent did - a session in production, or a run over a fixed test set - and scores it against a criterion: did it complete the task, was the answer grounded, did it call the right tool, did it follow the refund policy. The scorer can be code, an LLM judge or a person. Offline evals run before a release on a frozen dataset; online evals score production sessions after they finish. Either way, the eval happens after the behaviour, and its output is a number and a reason.

A guardrail is a control. It sits on the request path and makes a decision before something happens: allow, block, redact, or redirect with guidance. Its output is not a number but an outcome - the prompt was rejected, the response was masked, the command did not run. A guardrail that only logs is really an eval that happens to run early.

So the two differ in timing and in consequence. An eval is allowed to be slow, fuzzy and occasionally wrong, because the worst a wrong score does is mislead a chart. A guardrail has to be fast and precise, because a wrong decision either lets the damage through or blocks legitimate work in front of a user.

Side by Side

Read the table as two tools with opposite strengths, not a winner and a loser. Evals are good at everything that needs judgment and can wait. Guardrails are good at the narrow set of things that can be decided precisely and cannot wait.

Two Kinds of Guardrail, in Two Places

"Guardrail" covers two different mechanisms that sit in different places, and it matters which one you mean.

• Model I/O guardrails inspect what goes into and comes out of the model: prompt-injection attempts, personal data in a prompt, toxic or off-topic output, a response that leaks a system prompt. They sit between your application and the model, often as a proxy or gateway, or as an SDK call around each model request.
• Tool-call policies inspect the action the agent is about to take: the shell command, the file path, the SQL, the API call with its arguments. They sit at the agent harness's hook layer, between the model deciding to call a tool and the tool running.

They see different things. A model I/O guardrail sees the text going into and out of the model, and it can stop the model saying something harmful. Some gateways also inspect the tool calls a model requests, for traffic routed through them. Even then, the check sees the model asking for a command, not what the harness finally executes on the machine, and an action that never passes through the gateway is invisible to it. A tool-call policy at the hook layer sees the command the harness is about to run, and nothing about whether the agent's prose was polite. For agents with real permissions - shells, databases, repositories, payment APIs - the tool call is where irreversible damage happens, so it is the point a guardrail most needs to cover.

Where Evals Are the Right Tool

Evals win whenever the failure needs judgment or has not been named yet. "The agent gave a wrong but plausible answer", "it stopped escalating hard tickets", "it said the refund went through when it had not" - none of these can be written as a rule that runs in milliseconds before the action. An LLM judge reading the finished session can catch them, and a review over many sessions can catch the ones no single session reveals.

Evals are also the only way to know whether things are getting better or worse. A guardrail tells you how often it fired, which rises both when the agent gets worse and when more people use it. A pass rate per agent and environment, over time, is what tells you a prompt change helped or a model upgrade hurt. And evals are the safe place to be uncertain: a judge that is wrong one time in ten is a usable instrument, while a guardrail that is wrong one time in ten is an outage.

Where Guardrails Are the Right Tool

Guardrails win when the action is irreversible and the bad version can be described precisely. Deleting rows with no WHERE clause, force-pushing to main, reading a .env file, piping curl into sh, sending a secret in a tool result. For these, a score that arrives after the run is a post-mortem. The only useful answer is to stop the action before it executes, with a message the agent can act on. A rule like that is a few lines of code at the hook layer:

// sql-policies.js - a failure the evals found, now stopped before it runs
import { customPolicies, allow, deny } from "failproofai";

customPolicies.add({
  name: "block-unscoped-delete",
  description: "Deny DELETE statements that have no WHERE clause",
  match: { events: ["PreToolUse"] },
  fn: async ({ toolName, toolInput }) => {
    if (toolName !== "Bash") return allow();
    const command = String(toolInput?.command ?? "");
    if (/\bDELETE\s+FROM\s+[\w.]+\s*(;|$|"|')/i.test(command)) {
      return deny("DELETE without a WHERE clause is blocked. Add a WHERE clause or write a migration.");
    }
    return allow();
  },
});

That is a Failproof AI custom policy, written in the policy editor or installed locally with failproofai policies --install --custom ./sql-policies.js. It runs at PreToolUse, before the tool executes; blocking a tool call before it runs is verified on all twelve supported harnesses, Claude Code, Codex and Cursor among them. The regular expression is deliberately narrow: a guardrail that is too broad blocks legitimate work, and people turn it off. warn-destructive-sql, in the maintained coding agent pack, covers the wider class with a warning rather than a hard stop.

One Failure, Both Approaches

Take a data-cleanup agent with shell access to a database client, which one day deletes every row in a table instead of the stale ones.

1. With evals only

   A judge scoring each session against "only stale rows are deleted" flags the session after it finishes. An alert fires when the score drops. Someone restores the table from a backup, finds the prompt ambiguity, fixes it, and adds the case to the regression set. The measurement worked. The table was still deleted.

2. With a guardrail only

   A policy denies any DELETE without a WHERE clause. This failure is stopped, and the agent is told why. But the next failure - deleting the right rows from the wrong table, with a perfectly valid WHERE - passes the rule, and nothing measures it.

3. With both

   The policy stops the unscoped delete before it runs. The evaluator keeps scoring every session, catches the wrong-table deletions the rule cannot describe, and those findings become the next, narrower rule. The policy's hits become evidence in the next audit, beside the evaluation results, so you can see whether it is firing on real mistakes or on legitimate work.

Evals First, Then Guardrails, Then Evals Again

In practice the two form a loop, and the order of adoption follows the order of discovery. You cannot write a precise guardrail for a failure you have not seen, so evaluation comes first: score sessions, review populations, find what actually goes wrong. The failures that turn out to be repeatable and checkable become guardrails. Everything else stays a score. Then you keep evaluating, both to find the next failure mode and to check that each guardrail is catching real mistakes rather than blocking good work.

Failproof AI is built around that loop, and it is fair to say where its edges are. Evaluations run in the cloud and score each finished session: code checks, LLM judges, and an eval suite you already run, brought in as it is. Audits combine trace evidence, evaluation results and policy hits into findings, each with an analysis, a recommendation, a severity and the affected sessions, and a finding becomes an issue. From an issue, generate policy drafts a policy into the editor; nothing is published automatically. Backtest, in the dashboard, replays the draft against calls your fleet already made and counts the working calls it would interrupt, and the policy is deployed in observe mode before enforce. What it does not ship: judge models of its own, or content scanners for model input and output. If your main risk is what the model says rather than what the agent does, a model I/O guardrail is the tool built for it.

One middle ground is worth knowing. Between allow and deny, a policy can instruct: let the action continue, with guidance for the agent, where the harness supports it. That is useful for "probably wrong" cases where a hard block would cost too much. It is not a safety boundary, so keep irreversible actions on deny.

What Each Costs to Run

Eval cost is mostly judge tokens, and it scales with the number of sessions judged, the calls per session and the transcript length. It is off the hot path, so it never slows the agent, and it can be cut hard with code checks, sampling and trimming, as the judge cost guide shows. Guardrail cost depends on the kind. A rule that pattern-matches a command adds very little. A guardrail that calls a model before every action adds that model call's latency and tokens to every step of every session, which is why model-based guardrails are usually reserved for the few actions where the stakes justify them.

The cost people forget is false positives. An eval that misjudges a session costs a bad data point. A guardrail that misjudges an action stops a user's work, and a guardrail that does it often gets disabled, which leaves you with neither. Measure your guardrails' denials the way you measure anything else: sample them, read them, and keep the rules narrow.