64% → 71% on T-Bench 2.1 — with the same model.See the research →
← all events / steering-agents-with-policiesshare x · linkedin ·
● registrations openengineering sessiononline

steering agents with policies

an engineering session on writing policies that steer agents at runtime: from a failed trace to the policy that catches it, why the wording matters, and the policies that went wrong.

most agent failures aren't the model being bad at the task. the agent is missing one fact at one moment: the server dies when the shell closes, the file it's about to overwrite is the only copy, the output is right except for one byte.

steering agents with policies poster

what it's about

most agent failures aren't the model being bad at the task. the agent is missing one fact at one moment: the server dies when the shell closes, the file it's about to overwrite is the only copy, the output is right except for one byte. a policy that notices that moment and tells the agent what it's missing changes the outcome, without touching the model.

nikita agarwal, co-founder and ceo of failproof ai and co-author of the fire paper, walks through how we write policies that steer agents at runtime, and what we learned from reading thousands of failed runs. this is an engineering session, not a product demo.

  1. 01from failed run to policy

    how we read failed traces, find the mistake that keeps coming back, and turn it into a policy. with real examples from coding agents.

  2. 02allow, deny, instruct

    when to block a tool call, when to let it through, and when to send the agent an instruction instead. why instruct ends up doing most of the work.

  3. 03why the wording matters

    in our experiments a specific instruction at the right moment beat a generic "verify your work" at the same moment. what a good instruction looks like, and what a bad one looks like.

  4. 04when policies go wrong

    policies that fire too often, policies that never match reworded tasks, and a policy set that made runs almost 50% more expensive. what we changed.

  5. 05writing your first one

    a short live example of writing and testing a policy against a real failure, so you can do the same on your own agents.

  6. 06your questions, live

    the last 20 minutes are open. bring the failure your agents keep hitting.

good forengineers running agents in productionanyone building an agent harnessanyone who wants reliability without retraining

how the day runs

all times pdt · subject to change
  1. 15 min
    why agents fail and where policies fit

    the shape of a real failure: one missing fact at one moment, and the point in the run where telling the agent about it still changes the outcome.

    talk
  2. 25 min
    from traces to policies, with real examples

    reading failed traces, picking the mistake that keeps coming back, choosing between allow, deny and instruct, and writing the instruction the agent actually follows.

    talk
  3. 20 min
    open q&a

    bring the failure your agents keep hitting.

    social

who's on

  • SPEAKERNikita Agarwalceo & co-founder @ failproof ai

    co-author of the fire paper, and has read thousands of failed agent runs to write the policies that catch them. walks through the ones that worked, the ones that misfired, and the wording that made the difference.

    xlinkedin
  • HOSTNivedit Jaincto & co-founder @ failproof aixlinkedin
  • HOSTSahar Morbond ai

questions

something else? ask on discord →
do i need to use failproof to get anything out of this?

no. the session is about writing policies that steer agents at runtime, and failproof is where we run them.

reading a failed trace, picking the moment that matters and wording the instruction are the same problems on any harness.

what will i actually see?

real failed runs and the policies written against them, not a diagram of an architecture.

including the ones that went wrong: policies that fired too often, policies that never matched a reworded task, and a policy set that made runs almost 50% more expensive.

will i be able to write one afterwards?

yes. there is a short live example of writing and testing a policy against a real failure, end to end.

the paper behind it is fire: failure-informed runtime engineering for reliable language-model agents, arxiv:2609.26048.

can i bring my own problem?

yes. the last 20 minutes are open, so bring the failure your agents keep hitting.

the q&a starts at 6:40 pm pt.

how does registration work?

register on luma. it is free, and you are in straight away — no host approval to wait on.

it is online, so luma sends the joining link with your registration.

what is failproof ai?

failproof is your agent's oversight layer, steering it towards success. it traces your agents, finds failure patterns over time and sets up rules to prevent them from happening again.

the open-source cli is free: npm i -g failproofai.

MORE EVENTS

all events →

your room.
our agents. one night.

contact us