Does Judgment Labs block agent actions?

What Judgment Labs Documents

Judgment Labs' public site and docs do not describe intercepting, pausing or rewriting an agent's action. Everything documented acts on traces after they are recorded:

• Monitoring "applies known behaviors to live agent traffic" and "shows how often a signal appears". The responses it lists are "a notification, webhook, dataset action, or additional evaluation". (docs)
• Coverage is sampled. The same page says the measures "describe traces the judge evaluated, not necessarily every trace sent to the project".
• Automations run on finished traces. They "evaluate conditions against each completed trace", and code-side evaluation runs "without blocking". (docs)
• Scoring is off the request path. The judgeval README describes online monitoring that scores "live production traffic server-side with no latency impact". (GitHub)
• The coding-agent plugin observes. The judgment-plugin repository sends Claude Code and Codex sessions to Judgment Labs for judging; it does not gate commands.

So the precise answer: Judgment Labs tells you, after the fact and for the traces it judged, that an agent did something. It does not stand between the agent and the tool.

Why Monitoring and Blocking Use Different Checks

Judgment Labs keeps its judges off the request path, which gives its monitoring no latency impact but also means it cannot stop the action being evaluated. That is one risk posture: preserve agent speed and availability, then detect problems after execution.

Failproof AI takes a different approach. Code-based and LLM-based evaluations find failure modes after sessions run, while small deterministic policies handle the actions that need an immediate decision. Those policies run at the agent hook with very low latency, without waiting for an LLM judge on every tool call. Teams choose whether a policy should fail open or fail closed based on the consequence of allowing the action.

What Blocking Requires

A block needs three things: a decision point that runs before the action, visibility into what the action is, and a check fast and deterministic enough to sit there. There are three places to put it:

Choose by where your agent's risky actions happen. A coding agent's rm -rf, git push --force or terraform apply never passes through a model gateway, so only a harness hook sees it. A custom agent built on a framework you control can be guarded in code. An agent whose tools are all API calls behind a gateway can be governed there.

How to Add Blocking Next to judgeval

You do not have to replace Judgment Labs to add runtime control. Keep judgeval tracing and your existing evaluations, then add a policy at the agent hook for a specific action the evidence shows should be stopped. Failproof AI supports twelve harnesses, including Claude Code, Codex, GitHub Copilot CLI, Cursor, OpenCode, Goose, Hermes and OpenClaw, and can deny a supported tool call before it runs.

1. Install the CLI, wire the hooks, add the pack

   The CLI is free and MIT-licensed. policies --install wires hooks into your agent CLIs and enables no policy by itself. The coding agent policy pack holds 38 policies with 10 on by default - block-push-master, block-env-files, protect-env-vars, block-sudo, block-curl-pipe-sh and five sanitize-* redactors. Others in the pack, such as block-rm-rf, you enable by name.

   npm install -g failproofai
   failproofai policies --install
   failproofai policies add FailproofAI/policies
   failproofai policies add block-rm-rf

2. Write a policy for the behavior your judge keeps flagging

   Suppose your judge flags sessions where the agent ran terraform apply against production. A PreToolUse policy denies the command before it runs, and the deny reason tells the agent what to do instead. The file name must end in policies.js, policies.mjs or policies.ts.

// infra-policies.js - production infrastructure changes go through review, not the agent
import { customPolicies, allow, deny } from "failproofai";

customPolicies.add({
  name: "block-prod-terraform-apply",
  description: "Deny terraform apply or destroy against production",
  match: { events: ["PreToolUse"] },
  fn: async ({ toolName, toolInput }) => {
    if (toolName !== "Bash") return allow();
    const command = String(toolInput?.command ?? "");
    if (/\bterraform\s+(apply|destroy)\b/i.test(command) && /\bprod(uction)?\b/i.test(command)) {
      return deny("Production infrastructure changes go through a reviewed PR. Run terraform plan and describe the change instead.");
    }
    return allow();
  },
});

Enable it with failproofai policies --install --custom ./infra-policies.js. If a hard stop is too blunt, return instruct(reason) instead of deny(reason): the action continues with guidance for the agent - useful, but never a safety boundary. On Failproof AI Cloud you can author the same rule in the policy editor, backtest it against the calls your agents already made to count the working calls it would interrupt, and deploy it in observe mode before switching to enforce.

Keep the original evaluation running after deployment. Policy decisions show whether the risky action was attempted and stopped; the evaluation shows whether the agent learned to complete the task safely instead of repeatedly hitting the rule. Turn judge findings into runtime policies walks through discovery, testing, rollout and verification.

When Blocking Is Useful

Blocking matters when a single tool action can change production data, move money, expose credentials, publish code or send a customer message. Read-only agents and disposable sandboxes usually need stronger evaluation and monitoring before they need a runtime deny. Match the control to the consequence: observe low-risk behavior, steer recoverable mistakes and fail closed on actions the agent must never take.