what is a jev noul?

What the Noul Value Means

A result near one favors yes, a result near zero favors no, and a result near 0.5 means the two outcomes have similar modeled probability. It does not describe intensity. “Is the customer frustrated?” and “How frustrated is the customer?” are different questions; the first can be a Noul and the second needs an ordered Score rubric.

Optional true and false criteria can define both sides of the boundary. Keep them aligned with the instruction. A reversed or contradictory definition makes the instrument harder to interpret and is a documented Jev 1.13 weakness.

Use Multiple Nouls for Independent Labels

When a message may be both a refund request and a complaint, ask two Nouls. A Choice would force the labels to compete and select only one. Each Noul can have its own threshold because the consequences and base rates may differ.

Interpret the Number as Probability of Yes

For the question “Does the final reply claim that a refund was created?”, 0.82 means the model assigns 0.82 probability to yes under this state and question. It is not 82% task completion, severity or general confidence. A different Noul—such as “Did the refund tool succeed?”—would measure a different condition and should usually be replaced by an exact tool-result check.

Application code can derive an action from the value, but the action is not part of the model response. The same 0.82 could trigger an automatic tag in a low-risk workflow, human review for a customer-impacting claim, or no action where the evidence is incomplete. The threshold guide derives bands from error cost rather than attaching a universal meaning to one number.

{
  "noul": 0.82
}

Example: Check One Claim Against Supplied Evidence

The Noul answers only the semantic question about what the reply claims. Code still checks the tool result exactly. Keeping those two checks separate reveals a useful failure: the agent may claim completion even though no successful tool event exists. The agent-trajectory guide shows how to preserve that evidence link.

Do Not Round Noul into Truth Too Early

• Store the raw value and the resolved model and question version.
• Fit action thresholds against held-out labels from the target workflow.
• Keep a review band when the cost of a wrong action is meaningful.
• Do not expect separately asked positive and negated Nouls to sum to one.

Write One Direct Condition and Test Its Boundaries

A Noul should correspond to one label a reviewer can assign from the same evidence. Define positive, negative and unscorable examples before tuning wording. If reviewers disagree because the condition contains two decisions or missing policy context, changing the threshold will not repair the instrument. The question-writing guide covers atomic criteria and leakage-resistant iteration.

Validate Probability and Operating Threshold Separately

Calibration asks whether events assigned similar probabilities occur at similar rates across a population. It does not certify an individual 0.82 prediction. scikit-learn documents reliability diagrams and proper scoring metrics; the Jev calibration guide adds split design, bootstrapping and drift controls for this interface.

1. Sample representative positive, negative, rare and missing-evidence cases.
2. Obtain labels without showing reviewers the Jev result.
3. Freeze the model, Noul wording, criteria and state projection.
4. Group predictions into probability bands and compare predicted with observed positive rates.
5. Report Brier score, precision and recall at candidate thresholds, plus automation coverage and review volume.
6. Repeat by language, source, customer or other consequential slice and after every instrument change.

Common Noul Mistakes Change the Question Being Measured

Test paraphrases, hard negatives, missing evidence and adversarial text before deployment. Store the exact question and state-projection version with every result. The dataset guide explains label construction, and evaluate the evaluator covers calibration and slice errors.

• Combining two conditions with “and,” then treating one failure as evidence about both.
• Using a Noul for severity when the real answer needs ordered levels.
• Treating missing evidence as a confident no instead of an unscorable case.
• Writing positive and negative criteria that overlap or contradict each other.
• Copying one threshold across languages, model versions or consequences.