jev vs rules

Ask Whether Code Can Know the Answer Exactly

Run Deterministic Boundaries Before Semantic Judgment

A high Jev probability cannot grant a permission absent from the allowlist. A low probability should not override a mandatory statutory rule. This ordering limits both accidental errors and adversarial attempts to persuade the model.

Rules Fail Differently from Semantic Models

Rules are predictable but can become a brittle patchwork when language is expressed many ways. Jev can generalize across wording, but it introduces probabilistic errors, provider dependency and version drift. Count rule exceptions, false matches and change frequency alongside Jev errors, review load and cost.

Do not replace a long ruleset solely because it is long. Some complexity represents real policy. First separate exact constraints, taxonomic lookup and semantic interpretation; only the last category is a plausible Jev target.

Worked Example: Screen a Proposed Refund Action

This ordering prevents Jev from answering questions the database already knows and prevents a semantic result from creating authority. It also makes failures diagnosable: a bad date check is a rule defect, missing conversation evidence is a projection defect, and disagreement with reviewed intent labels may be a Jev instrument defect. The agent tool-call evaluation guide uses the same evidence separation.

Determinism Is Especially Valuable at Trust Boundaries

TypeSafe documents adversarial steering as a Jev 1.13 weakness. Untrusted text can influence a semantic result, so use sandboxing, least privilege, argument validation and output encoding regardless of the score. Rules also need canonicalization because attackers exploit encodings, path aliases and parser differences.

Build adversarial fixtures for both layers. A rule test should prove equivalent encodings resolve identically; a Jev test should cover instructions embedded in user content and retrieved documents. The limitations guide supplies model-specific cases.

Compare Total Maintenance and Error Cost

Rules have engineering and review cost even when inference is free. Jev adds request cost, latency, outages and monitoring. Estimate the frequency of policy changes, traffic volume, false allows, false blocks and human escalation before choosing.

At low volume, a human or a clear rule may be simpler. At high volume with stable labeled data, a trained classifier may be cheaper. The alternatives guide frames these options without assuming Jev is always the answer.

Test the Hybrid, Rules-Only and Model-Only Designs

A model-only design is an important negative control even when it should never ship: it reveals which exact constraints the model happens to mimic and where it fails. A rules-only baseline reveals whether semantic inference adds measurable value at all. The benchmark methodology supplies paired analysis and uncertainty reporting.

1. Freeze representative cases and independent labels, including exact denials and semantic edge cases.
2. Run the current ruleset, Jev-only candidate and layered hybrid on the same evidence.
3. Report false allows and false blocks separately for deterministic and semantic conditions.
4. Measure automation coverage, review volume, p95 latency, provider failures and complete cost.
5. Perturb encodings, wording and untrusted content to test both canonicalization and model steering.
6. Promote the smallest architecture that meets the stated risk target and retain rollback fixtures.

Log Which Layer Made the Decision

• Normalized input and deterministic rule version.
• Authorization and allowlist result before Jev ran.
• Jev state, question and resolved model references.
• Full distribution and code-owned threshold band.
• Final action, fallback, override and eventual outcome.