jev choice explained

Choice Is Relative to the Candidate Set

A Choice distribution compares only the options supplied. If the correct route is absent, Jev must still distribute probability over the remaining candidates. Add other, none or needs_review when that is a valid outcome, or pair Choice with a Noul that asks whether any candidate is suitable.

Descriptions should make the distinctions explicit. Candidate names alone may work for familiar labels, but domain-specific boundaries belong in criteria. Code should filter candidates that are unavailable or unauthorized before the request.

Interpret the Distribution and Confidence Together

Per-option probabilities show the relative alternatives. Confidence summarizes how concentrated that distribution is. Several reasonable options can produce low confidence without making the task erroneous. Conversely, a concentrated distribution does not prove the state was complete or the candidate set was valid.

Read the Selected Key Without Discarding the Alternatives

The selected key is billing, but the response is not equivalent to a deterministic rule that proved billing. The 0.31 assigned to technical support shows a meaningful alternative, while the low other probability only compares the explicit options in this request. It cannot prove that the taxonomy contains every real destination.

Store the complete probability map, the candidate definitions and the resolved model—not only billing. That record lets a team change a review threshold, inspect near-ties and replay historical decisions without calling the model again. The confidence-versus-probability guide explains why the separate confidence field is not the probability that the selected label is correct.

{
  "choice": "billing",
  "probabilities": {
    "billing": 0.61,
    "technical": 0.31,
    "account": 0.06,
    "other": 0.02
  },
  "confidence": 0.30
}

Design Candidates as a Taxonomy, Not a List of Names

Choice works best when candidates answer the same question and only one should be selected. If a ticket can be both billing-related and urgent, those are separate dimensions: use Choice for route and Score for urgency. If it can be both a complaint and a refund request, use separate Nouls. The Choice-versus-Noul comparison shows the same input modeled both ways.

Criteria should state positive boundaries and important exclusions in plain language. Avoid encoding eligibility, permissions or live capacity inside prose when code already knows those facts. Filter the candidate set first, then ask Jev which remaining candidate best fits the semantic evidence.

Example: Route Only Among Handlers That Code Permits

This separation prevents semantic similarity from creating permission. Jev may prefer a billing agent, but code must first establish that the agent is available and allowed to see the account. The agent-routing guide develops this architecture.

Good Choice Uses

• Route a request to one available handler.
• Select one relevant passage from a retrieved shortlist.
• Pick one approved tool or agent skill.
• Assign one primary failure category for reporting.
• Choose a bounded function while code validates its arguments.

Choose an Explicit No-Match Design

Do not use low confidence as a universal no-match detector. A distribution can be concentrated around the least-bad option even when no candidate fits, or diffuse when several valid candidates are genuinely similar. No-match and ambiguity are different conditions and should be measured separately. The abstention guide explains how review bands affect both risk and coverage.

Evaluate Choice as a Full Distribution

Top-choice accuracy is necessary but incomplete. Inspect the confusion matrix, probability assigned to the correct option, margin between the top two candidates, no-match recall and performance by class. A routing model can appear accurate overall while consistently sending a rare high-impact category to the wrong owner.

Changing the candidate set changes the comparison. Adding an option, editing criteria or filtering unavailable choices creates a new evaluator version that needs replay. Use regression testing and retain the full distribution for threshold analysis.

Run a Routing Experiment That Can Reveal Costly Mistakes

A confusion matrix makes directional errors visible: billing routed to technical may be cheap, while a security incident routed to general support may be unacceptable. scikit-learn’s classification-metrics documentation defines the standard calculations; the Jev benchmark methodology adds grouped splits, uncertainty, versioning and complete-system cost.

1. Sample tickets from the deployment population, including rare routes and genuine no-match cases.
2. Have reviewers label the correct route and record disagreements before seeing Jev output.
3. Freeze the state projection, criteria, candidate set, model and provider.
4. Report the confusion matrix and per-class precision and recall, not only overall accuracy.
5. Measure no-match recall, top-two margin, calibration, latency, failures and review volume.
6. Compare with the existing rules, classifier or LLM router on the same evidence and labels.