when not to use jev

Jobs That Belong Elsewhere

Keep Five Hard Boundaries Outside Jev

Jev may evaluate a semantic fact used after these checks—for example, whether the user’s message requests a refund—but the identity, amount limit and actual ledger write remain deterministic. OWASP’s excessive-agency guidance similarly emphasizes minimizing permissions and requiring authorization in downstream systems rather than trusting model output.

The Jev-versus-rules guide shows the ordering: normalize, deny exact failures, restrict to allowed candidates, ask the semantic question, then let application code decide and log.

Warning Signs in a Proposed Jev Design

• The team cannot list the possible answers before calling the model.
• The question contains “and” because several independent decisions were combined.
• A wrong high-confidence result has no safe fallback or override.
• No representative labels or downstream outcomes exist for evaluation.
• The request sends an entire transcript although the decision needs three fields.
• A model threshold is being used as identity, authorization or legal approval.

Recognize Architectural Anti-Patterns Before Benchmarking

Use This Replacement Decision Tree

Several branches can apply in one workflow. Code may retrieve records, an LLM may draft an answer, Jev may check one bounded claim, and a human may approve an exceptional action. The alternatives guide compares these components directly.

Keep Jev for the Semantic Remainder

Rejecting Jev for one part of a workflow does not reject it for all parts. Code can find candidate dates and validate them while Jev selects which mentioned date answers the user’s question. A permission system can filter actions while Jev selects the most relevant permitted handler. A generative model can draft text while Jev evaluates one bounded criterion.

The boundary should be visible in code and evaluation. That makes failures attributable: missing evidence, wrong semantic judgment, faulty threshold, deterministic bug or service outage.

Replace the Unsuitable Step, Not Necessarily the Whole Pipeline

This decomposition avoids asking any model to do database work and prevents a generated explanation from becoming authorization. It also creates separate tests: retrieval recall, arithmetic unit tests, generation faithfulness, Jev calibration, permission tests and reviewer quality.

Convert Documented Weaknesses into Test Cases

These are version-specific warnings in TypeSafe’s Jev 1.13 documentation. A later version may change behavior, but migration still requires replay. The limitations page tracks the exact failure taxonomy and mitigations.

Do Not Make Jev the Sole Control for High-Impact Decisions

A well-performing evaluator can support triage, evidence review or monitoring in legal, employment, medical, financial and physical-safety workflows. It should not become the only basis for an irreversible decision merely because its distribution is concentrated. These settings require domain-specific governance, qualified review, appeal, audit and deterministic constraints.

Separate assistance from authority. Jev can prioritize documents for a lawyer, but does not deliver legal advice; it can flag a potentially risky transaction, but does not transfer funds; it can classify a smart-home request, but must not bypass device safety interlocks. The relevant legal, finance, recruiting and smart-home guides spell out those boundaries.

Use a Stop Test Before Adding Jev to Production

“Interesting output” is not an adoption criterion. A useful Jev component must improve decision quality, safe automation coverage, latency, cost, review effort or maintainability against the current baseline. Preserve negative results; the reranking field report shows why a new model should not be kept when the measured pipeline does not improve.

1. Write the bounded answer space and the exact evidence available at decision time.
2. Implement the simplest deterministic, retrieval, classifier or human baseline that can solve the same job.
3. Create blind development, calibration and held-out test sets with consequential slices.
4. Measure class errors, calibration, automation coverage, latency, failures and total workflow cost.
5. Stress documented Jev weaknesses and missing-evidence behavior.
6. Stop if Jev does not improve a predeclared product metric enough to pay for its added dependency.

A Valid Task Can Still Be an Uneconomic Use

A low-volume decision with clear rules may not justify an external model dependency. At very high stable volume, a trained classifier may offer lower unit cost. A human may remain cheaper when cases are rare, complex and already require accountable review. Include integration, retries, monitoring and mistakes in the comparison.

Run a small offline benchmark before building the full path. If Jev does not improve accuracy, coverage, latency, review effort or maintainability over the simplest baseline, do not add it. Use the benchmark methodology and pricing guide to make that decision explicit.

Record Rejected Use Cases as Engineering Decisions

A short decision record prevents the same unsuitable proposal from returning under a new name. It also leaves room for a valid semantic subproblem later without weakening the exact boundary. Revisit the record only when the task, evidence, model version or alternatives materially change.

{
  "candidate_use": "calculate_refund_total",
  "decision": "do_not_use_jev",
  "reason": "exact arithmetic over structured line items",
  "replacement": "tested decimal calculation in application code",
  "possible_jev_role": "none",
  "review_if": "the task changes to interpreting an ambiguous refund request"
}