← all posts
post №21·Nov 17, 2026·10 min read

we made an ai agent play five games. then we gave it policies.

The same agent played Pokémon, Civilization, poker, Yu-Gi-Oh! and DOOM twice: once bare, once with runtime policies on its tool calls. How we wired real game engines to an agent, and what a policy can and can't change, measured on held-out games.

A runtime policy sees all events in the harness, including the tool calls an agent is about to make. It can veto a call with a reason, or add a fact to what comes back. The question is whether that actually changes how the agent plays out a task. To measure it, we put the same agent in five real games and ran each one twice: once bare, once with failproof ai policies on its tool calls.

Same model, same prompt, same tools, same harness. The only difference between the two arms is a hooks config.

GameBare agent+ failproof ai
PokémonWin rate18%42%
CivilizationGame score224244
PokerBig blinds per 100 hands−10.8+1.5
Yu-Gi-Oh!Misplays per duel1.270.45
Held-out games, frozen policies, same model, prompt and tools in both arms.

Why games

Every decision in a game is a tool call with a checkable context, and every game ends in a number: a win, a score, chips won or lost. And the agent's mistakes are the ordinary agent mistakes: forgetting a step before ending its turn, guessing at arithmetic it could have computed, retrying an action the environment already rejected.

Instead of a paper on runtime steering, five games.

We picked turn-based games with open engines, so every decision has a public fact a policy can check:

GameEngineOpponent
Pokémon, Gen 9 random battlesPokémon Showdown server, offlineFoul Play, an open-source tree-search bot
Civilization, 60 turnsUnciv, the open-source Civ V remakeUnciv's own AI civilizations
Heads-up no-limit hold'emPokerTHPokerTH's computer player
Yu-Gi-Oh!EDOPro duel serverWindBot
DOOMViZDoom, Freedoom mapsThe monsters

The setup

Each run gets its own container and its own failproof ai daemon. In the policy arm the run directory carries a hooks config, so every tool call goes through the policies before it executes, and every result comes back through them. The bare arm has no config. A run is voided if the policy arm records no hook events, or the bare arm records any.

two arms · one difference

Both arms run the same agent with the same model, prompt, tools and harness. In the bare arm every tool call goes straight to the game. In the failproofai arm the same call first passes through the hooks of a frozen policy pack, which can veto a bad call with a reason before it runs, or add an exact fact to what the game returns.

A policy can add a fact after a tool call returns, warn before an action, or veto it. It never picks the move. Every veto carries a reason and the agent chooses again; most vetoes also let the agent override by repeating the exact same call, so a wrong policy can't lock it out.

Two copies of the same agent. Policies drop a fact, raise a warning or block a bad move; the agent still decides.

One real call from the held-out Pokémon batch:

one veto, from a real held-out battle

From a real report battle. The agent tries to switch out while the opponent's Venomoth is boosted. The pk-free-turn policy denies the call before it reaches the server and tells the agent that Magnezone can hit Venomoth now, with Thunderbolt doing 415 to 488 percent, a knockout. The agent chooses again and uses Thunderbolt, which is allowed. The new state comes back with the next turn's damage table added as a fact.

Under the hood

None of these games has a seat for an agent. For each one we kept the engine and the opponent bot as their authors ship them, and replaced only the human seat with a bridge.

inside one game container

Inside one container per game: the agent session sends each tool call through failproofai's daemon and policies, then to the game tools, a Python environment that talks to a small bridge over a socket. The bridge drives the real game engine, which runs the unmodified rules and the open-source opponent bot. The environment writes exactly what it showed the agent to visible_state.json, and that file is the only game state a policy may read. The two arms differ only in whether the tool calls pass through failproofai.

  • Game tools. A Python environment per game exposes a few tools to the agent over MCP: battle_state, use_move, switch; table_state, act; end_turn and unit orders. Each returns a text view of what a human in that seat would see, and nothing from the engine's hidden state.
  • Bridge. Where the game has no API, a small patch on the human seat sends its view as newline-delimited JSON over a local socket and plays the reply through the game's own action functions. Each patch lives in the repo under the licence of the project it patches.
  • Observation file. Whatever a tool shows the agent is also written to visible_state.json in the run directory. That file is the only game state a policy may read, so "no hidden state" holds by construction.

A policy is a module that registers handlers on the hook events. This is the poker free-check veto from the repo:

customPolicies.add({
  name: "poker-free-check",
  description: "Hard-veto folding when checking costs nothing.",
  match: { events: ["PreToolUse"] },
  fn: safe((ctx) => {
    if (!isTool(ctx, "act") || String(ctx.toolInput?.action).toLowerCase() !== "fold") return;
    const st = visible(ctx)?.state;   // reads visible_state.json, nothing else
    if (st && st.to_call === 0)
      return deny("Nothing to call here — checking is free and keeps your cards. " +
                  "Folding now only gives up the pot. Choose again.");
  }),
});
Every game lands in the dashboard as a session: find the mistake the agent keeps making, then fix it with a policy.

How each game is wired:

  • Pokémon. The official Showdown server runs offline at a pinned commit. The agent's seat is a poke-env player whose move choice waits on our tools. Foul Play runs a Monte Carlo tree search over a Rust battle engine, unchanged apart from logging in to a local server; its search time per move sets its strength. Results are re-scored from the server's own replay files.
  • Civilization. Two new Kotlin files in Unciv start a headless game in which one civ is a human player driven over a socket and every other civ is the stock AI. The bridge exposes that player's empire, the tiles and units it can see, and the game's own battle previews and city-site suggestions. Four small guards stop the engine crashing without a window or sound system.
  • Poker. PokerTH is built from the Debian source with a patch and run under a virtual display. The human seat's view goes out as JSON, and the reply is played through the same functions the buttons call. Two environment variables seed the deck and swap the seats, which is what makes duplicate poker possible.
  • Yu-Gi-Oh! The Multirole duel server hosts two WindBot clients: the stock opponent, and ours, patched to send every decision plus the duel state the server sent that seat to our environment. The server enforces the rules and saves a replay of every duel.
  • DOOM. ViZDoom runs the DOOM engine as a library. After every action the agent gets a 320×240 screenshot plus text: health, ammo, its own position, the objects drawn on that frame with bearing and distance, and a seven-column wall-distance scan from the depth buffer. Kills, damage and exits come from engine counters the agent never sees.

Keeping it honest

The rules were written down before any report run:

  • No hidden state. Policies read only what the agent was shown.
  • Policies never choose. Facts, warnings and vetoes only.
  • Tune on dev, report on held-out. Dev rounds are the practice games we tuned the policies on; held-out games are fresh ones they never saw. After the dev rounds, each pack was frozen and hashed, and only held-out results are reported.
  • Pair what can be paired. Both arms play the same Civilization maps and DOOM levels. Poker uses duplicate decks: each deck is played twice with the seats swapped, so card luck cancels.
  • Score from the game. Metrics come from the engine, never from the agent's text. Crashed or interrupted games are voided and re-run, never dropped.

how a number in this post gets made

Each game goes through the same pipeline. First the opponent is calibrated so the bare agent loses a fair share of games. Then dev rounds: watch the agent play, find a repeated mistake, write or change one policy, and run again. Then the pack is frozen and hashed. Then a held-out report batch runs both arms on fresh seeds; any interrupted game is voided and re-run on its own. Finally scores are taken from the game's own records. Every number in the post is from the report step.

Results

All numbers are from held-out games.

Pokémon

Every turn is a damage calculation, and the bare agent was guessing: it knows roughly what is super effective, not whether a hit does 30% or 90%. The main policy runs the public damage calculator after every action and gives the agent the damage range of each legal move and switch, what Terastallizing would change, and late in the game, which of its Pokémon beats each of the opponent's. A few vetoes cover switching out of a free hit, a much weaker attack than another legal one, and setting up into a knockout.

Pokémon: the bare agent guesses the damage; with policies it sees the numbers before it commits.
100 + 100 battles vs Foul PlayBare+ failproof ai
Win rate18%42%
Own Pokémon left at the end0.431.64
Opponent Pokémon knocked out3.774.52

2.3 times the win rate, Fisher exact p = 0.0003.

Civilization

The bare agent's problem was bookkeeping. It kept ending turns with a city building nothing, no research queued, or a social policy it could have adopted. A check on every end_turn lists what is idle, next to the game's own score formula. Other policies reject city sites the game would refuse, and guard undefended cities and unescorted settlers.

Civilization: idle cities, no research and unspent policies, caught before the turn ends.
100 paired mapsBare+ failproof ai
Score at turn 60224.3244.1 (+8.8%)
Mistakes per turn0.990.04
Technologies9.811.8

The policy game beat its bare twin on 64 of 99 decided pairs (p = 0.002).

Poker

Fifty hands a session, heads-up. The bare agent's leak was passivity: folding strong hands, checking hands it should bet. The policy gives hand strength, equity against a random and a strong range, pot odds, and the opponent's tendencies counted from the public table log. One hard veto: never fold when checking is free.

Poker: from losing money to breaking even.
Held-out sessionsBare+ failproof ai
Big blinds per 100 hands−10.8+1.5
Strong hands folded, per 100 decisions0.500.21

Deck for deck over 102 duplicate decks, the edge is +9.7 big blinds per 100 hands, 95% interval −10.3 to +28.3. That crosses zero: a lean, not a result.

Yu-Gi-Oh!

The policies build checklists from the engine's list of legal moves before entering battle and before ending the turn, block skipping Main Phase 2 with summons still available, show a card's text the first time the agent uses or passes on it, and show the worst-case damage the opponent can deal next turn.

Yu-Gi-Oh!: a veto on ending the turn from the Battle Phase with Main Phase 2 still to play.
40 + 40 duelsBare+ failproof ai
Misplays per duel1.270.45
Win rate57.5%65%

Misplays fell by two thirds. The 7.5-point win gain is not significant at 40 duels per arm.

DOOM

The agent's real problem in DOOM is navigating 3D space from a small screenshot and text, which no policy fixes. The final pack is four vetoes: no firing with no enemy visible, aim before firing, don't repeat a move the game reported as blocked, no long bursts at low health with enemies close.

DOOM: the veto on firing with no enemy visible.
30 paired mapsBare+ failproof ai
Shots with no enemy visible, per run0.870
Blocked moves, per run12.38.7

Wasted shots went to zero (p = 0.001). Exits and survival didn't move.

What we learned

Every pack cut the mistake it targeted:

the mistake each pack targets, per gamebare agent+ failproofai
00.51.0civilization0.990.04yu-gi-oh!1.270.45doom0.870.00poker0.500.21

Lower is better. Civilization: mistakes per turn. Yu-Gi-Oh!: misplays per duel. DOOM: shots fired with no enemy visible, per run. Poker: strong hands folded, per 100 decisions.

Only some of those cuts became wins:

win rate, held-out gamesbare agent+ failproofai
0%25%50%75%100%pokémon18%42%yu-gi-oh!57.5%65%

Pokémon: 100 + 100 battles vs Foul Play, p = 0.0003. Yu-Gi-Oh!: 40 + 40 duels vs WindBot, not significant.

  • Exact facts at decisive moments win. Pokémon has many decisive decisions, each with a number the agent was guessing. That is where the win rate more than doubled.
  • Fixing a mistake isn't winning. Yu-Gi-Oh! misplays and DOOM's wasted shots collapsed, but they weren't what decided those games.
  • Dev gains shrink. Tune on one set of games, report on another:

dev rounds flatter you · held-out is the number

Gain of the policy arm over the bare agent. Civilization: dev round one +63.8 points, dev round two +49.7, held-out report +19.8. Poker: dev +56.8 big blinds per 100 hands, held-out +9.7. The dev bars are drawn dashed because they were measured on the same games used to tune the policies.

What broke along the way

ProblemEffect and fix
A monster name spelled two ways in DOOMZombies were invisible to the agent in both arms, and the blind-fire veto fired on real targets. The first batch was voided.
Host suspended mid-batch30-minute gaps between moves in both arms. Any game with an unexplained gap is now voided and re-run.
12 poker sessions played twiceAnalysis kept whichever copy it read last. It now keeps the first-played copy; the paired edge moved from +10.5 to +9.7.

The short version

A policy sees the tool call before it runs and the result before the agent does. With nothing else changed, that was enough to more than double a Pokémon win rate, add 8.8% to a Civilization score on the same maps, and cut every targeted mistake. The mistakes were the ordinary ones: skipped steps, guessed numbers, repeated failures.

Same agent, no retraining.

The harness, every policy pack and every scored game are at github.com/FailproofAI/failproof-arcade.

failproof ai enforces your policies at runtime, on every tool call →