jev as a runtime policy signal

A Safe Decision Order

A Jev probability is evidence about a semantic condition. It is not a permission token. If a user or system policy has not authorized an action, high model confidence cannot create that authorization. Likewise, an exact deny rule should not be softened because a semantic model prefers another answer.

1. Validate schemas, identities, permissions, paths, amounts and other exact invariants in code.
2. Build the smallest pre-action state that contains the evidence for one semantic question.
3. Ask Jev a Noul, Choice or Score question with explicit criteria and boundary cases.
4. Apply code-owned thresholds and vetoes based on the consequence of a mistake.
5. Choose allow, observe, review or block, then log the decision and later outcome.

Good and Bad Policy Questions

Define Failure Behavior Before the Happy Path

A synchronous policy must decide what happens when Jev times out, returns 429, is unavailable, receives missing state or produces a low-confidence distribution. Low-risk read actions may fail open with an audit event. Irreversible writes, money movement, external messages and permission changes may require review or fail closed. The choice belongs to the product threat model, not a universal default.

Jev 1.13 documentation also warns that adversarial state can steer the answer. Treat tool arguments, retrieved pages and user content as untrusted. Use narrow criteria, deterministic defenses, adversarial tests and least privilege. A Jev check is one layer, not a complete prompt-injection defense.

Backtest, Observe, Canary, Enforce

Start with completed historical calls and compute what the proposed policy would have done. This is a replay of the policy decision over recorded evidence, not a re-execution of the agent. Review false blocks, false allows, uncertain cases and missing state by consequence.

Next run the policy in observe mode on current traffic. Log its proposed action without changing the agent. After the measured operating point is acceptable, canary enforcement on the smallest safe slice. Keep overrides, rollback and model/question pinning available. Re-run the backtest before moving a Jev alias or threshold.

What the Audit Record Should Contain

• Trace and event reference, plus a hash or version of the projected state.
• Jev version, question and criteria version.
• Raw typed answer and full available distribution.
• Thresholds, deterministic vetoes and selected policy action.
• Timeout, retry and fallback path when the call did not complete normally.
• Human override and the eventual real-world outcome used for evaluation.

Jev Policies in Failproof AI

Failproof AI policies operate at the agent harness hook layer. The product workflow starts with production sessions and evaluations, groups repeated evidence into findings, and can turn a narrow preventable failure into a policy that is backtested and observed before enforcement. A Jev decision can provide the semantic signal within that lifecycle while Failproof owns policy state and action.

Jev policies are live in Failproof AI for beta users as of September 22, 2026. Email nikita@befailproof.ai for access. The exact supported hooks, state mapping, timeout behavior, billing and configuration are not documented on this page.