guardrails for on-call engineers using ai agents

Why on-Call Is the Riskiest Place to Run an Agent

On-call permission sets tend to be broad because nobody wants an incident blocked on access. They are also long-lived: IAM Identity Center sessions can last up to 12 hours, and many teams set the maximum for on-call roles. Put that next to an agent asked to “fix the failing deploy” at 3 a.m., and you have the highest blast radius, the least review and the most ambiguous instructions in the same shell.

Agents are genuinely useful during incidents: reading logs, correlating metrics, drafting a rollback. The goal is not to keep them out of on-call work. It is to keep diagnosis fast and keep a human hand on every production change.

Two Ways to Define the on-Call Persona

Most teams use both: the standing admin group’s machines always get the strict set, and the credential rule catches anyone who assumes the on-call role.

Policy: Agents Diagnose, Humans Change Production

Every AWS, kubectl, Terraform or Helm segment in the command has to start with a read-only call for the command to pass, so aws ssm get-parameter … && aws ec2 run-instances … is denied. Command substitution, backticks, comments and nested shells are denied outright under elevated access, because they hide what actually runs. The deny reason is written as a runbook step: the agent posts the command and rollback, and the engineer runs it.

import { customPolicies, allow, deny } from "failproofai";

const ELEVATED = /(?:--profile[=\s]+|AWS_PROFILE=)(oncall|admin|breakglass)[\w.-]*|--(?:kube-)?context[=\s]+\S*prod/i;
// Substitutions, comments, newlines and nested shells hide what runs; with
// elevated access they are denied rather than parsed.
const OPAQUE = /\$\(|`|#|\n|\b(?:bash|sh|zsh)\s+-c\b|\beval\b/;
// Read-only shapes per tool, anchored to the start of the segment (after
// sudo/env/time and VAR=value). Flags may come before the subcommand.
const START = String.raw`^(?:(?:sudo|env|time)\s+)?(?:[A-Z_][A-Z0-9_]*=\S+\s+)*`;
const FLAGS = String.raw`(?:\s+--?[\w-]+(?:=\S+|\s+[^-\s]\S*)?)*`;
const READ_ONLY = [
  new RegExp(START + "aws" + FLAGS + String.raw`\s+[\w-]+\s+(describe-|get-|list-|lookup-events|filter-log-events|tail\b)`),
  new RegExp(START + "aws" + FLAGS + String.raw`\s+(sts\s+get-caller-identity|s3\s+ls)\b`),
  new RegExp(START + "kubectl" + FLAGS + String.raw`\s+(get|describe|logs|top|events|explain|version|api-resources)\b`),
  new RegExp(START + "helm" + FLAGS + String.raw`\s+(list|ls|status|get|history|show|template|version)\b`),
  new RegExp(START + String.raw`terraform(?:\s+-[\w-]+(?:=\S+)?)*\s+(plan|show|output|validate)\b`),
];

customPolicies.add({
  name: "oncall-production-guard",
  description: "Elevated access: agents diagnose, humans change production",
  match: { events: ["PreToolUse"] },
  fn: async (ctx) => {
    if (ctx.toolName !== "Bash") return allow();
    const cmd = String(ctx.toolInput?.command ?? "");
    if (!ELEVATED.test(cmd)) return allow();
    if (OPAQUE.test(cmd)) {
      return deny("With elevated access, run one plain command at a time: no $( ), backticks, comments or nested shells.");
    }
    const segments = cmd.split(/&&|\|\||;|\|/).map((s) => s.trim()).filter(Boolean);
    const tools = segments.filter((s) => /\b(aws|kubectl|terraform|helm)\b/.test(s));
    if (tools.length > 0 && tools.every((s) => READ_ONLY.some((re) => re.test(s)))) return allow();
    return deny("This changes production with elevated access. Post the exact command and its rollback in the incident channel; the on-call engineer runs it.");
  },
});

Test Cases

With a Decision Model

The coding agent pack’s opt-in block-kubectl denies the whole CLI and is reviewable by the production-infra-change check. With Jev on and the FailproofAI/jev-policies pack installed, read-only calls like kubectl get are cleared while production mutations stay denied. Each review is a model call, so expect a short pause on reviewed calls.

Make It Robust with a Decision-Model Check

A pattern only catches the spellings it lists, so every new tool, wrapper or script needs another pattern. A decision-model check asks what the call does and whether the person asked for it, so it holds up when the wording changes. Keep the regex policy as a hard floor and add the check on top.

Keep the regex policy hard, so its denies are final, and add this check beside it for what the pattern cannot see. It is not user-overridable, so a prompt asking the agent to “just restart prod” does not clear it.

We write custom checks like this one for your own rules: talk to us and tell us what should never happen on your machines. Jev comes with Failproof; turn it on whenever you choose, with failproofai jev setup or a Cloud machine key that carries jev:evaluate.

import { semanticPolicies } from "failproofai";

semanticPolicies.add({
  name: "oncall-production-change",
  title: "Changed production from an agent session",
  appliesTo: ["shell", "network", "other"],
  mode: "deny",
  userCanOverride: false,
  probes: [
    {
      id: "changes_production",
      instructions: "Does this call change live production resources (deploy, scale, restart, fail over, delete, or modify data or configuration), rather than only reading them?",
      criteria: { true: "It changes production.", false: "It only reads, or targets a non-production environment." },
    },
  ],
  guidance: "Agents diagnose; humans change production. Post the exact command and its rollback in the incident channel for the on-call engineer to run.",
});

• python scripts/failover.py --region eu-west-1 or make rollback ENV=prod: no AWS, kubectl, Terraform or Helm segment for the pattern to inspect.
• MCP tool calls to a cloud or deployment service, which never pass through the shell.
• A production change written into a script the agent then runs.

Keeping Incidents Fast

• Write the deny reason as a runbook pointer. “Post the command and rollback; the engineer runs it” keeps the incident moving.
• Add narrow exceptions after observing. If agents routinely propose the same safe command, such as a rollout restart, allow that exact shape after a week in observe mode.
• Keep a break-glass path that is not an agent. The human can always run the command directly; the policy only gates the agent.
• Review after every incident. Open the session from Observe → Sessions and read which calls were denied. Adjust the policy, publish a new version, roll forward.