jev-latest vs jev-preview

The Aliases Match Today but Express Different Intent

TypeSafe’s model page currently points both aliases at jev-1.13.0. That is a dated routing fact, not a promise that the aliases move together. jev-latest represents the current stable channel; jev-preview represents the newest preview channel. A versioned identifier names one evaluated artifact.

Choosing an alias delegates upgrade timing to the provider. That can be convenient in a playground and risky in a policy whose threshold was calibrated on a particular distribution. Read the current Jev model card before treating this snapshot as current.

Use Different Identifiers at Different Lifecycle Stages

The recommendation depends on consequence, not a blanket ban on aliases. A disposable prototype can favor freshness; an audited evaluator needs repeatability. Keep the identifier in configuration rather than scattered through code so promotion changes one reviewed artifact.

A Model Change Can Invalidate More than Accuracy

A new model may change selected labels, probability calibration, confidence distribution, language slices, sensitivity to state length, latency and rate-limit behavior. Even a quality improvement can move the operating point: a threshold of 0.82 fitted on one version has no automatic statistical meaning on the next.

Version the complete measurement instrument: model, provider, question wording, criteria, state projection and threshold. If two of these change in one release, attribute regressions with a factorial test or change them sequentially. The confidence guide explains why preserving distributions is essential for this analysis.

Pin the Complete Evaluator Bundle

This is an application manifest, not a TypeSafe configuration file. Pinning only the model leaves other behavior-changing inputs implicit. Hash or version the question definition, candidate labels, state projector and threshold logic so a result can be reconstructed later.

Record requested and resolved model separately. A pinned request should resolve to itself; an alias request records both the channel and the artifact that actually served it. If the provider does not expose a resolved upstream version, record that absence rather than inventing one.

{
  "evaluator": "refund-requested@v4",
  "provider": "typesafe-direct",
  "requested_model": "jev-1.13.0",
  "resolved_model": "jev-1.13.0",
  "primitive": "noul",
  "question_version": "refund-requested-v4",
  "projection_version": "support-trace-v2",
  "threshold_version": "refund-bands-v3",
  "client_version": "<pinned-sdk-version>",
  "approved_dataset": "refund-heldout-2026-09"
}

A Safe Alias-to-Pinned Upgrade Pipeline

1. Record the alias target, provider, SDK, timestamp and model returned by every response.
2. Gate on task-specific errors and calibration, not aggregate agreement alone.
3. Inspect disagreements manually, especially high-consequence and minority slices.
4. Set rollback triggers for error rate, review volume, latency, cost and service failures.

Change One Layer at a Time When Possible

Use component comparisons for diagnosis and a whole-bundle comparison for the release decision. A factorial design can estimate interactions when several changes must ship together, but it multiplies runs and analysis. At minimum, do not call every changed outcome a “model regression” when the question or state changed too.

Compare Versions on Decisions, Distributions and Operations

Use paired results on identical examples. Report the action transition matrix—old allow to new review, old review to new allow, and so on—because average accuracy can stay unchanged while operational behavior moves substantially. Inspect every transition involving a high-consequence slice.

Thresholds belong to a model-and-instrument bundle. Fit candidate thresholds on calibration data, freeze them, then estimate performance once on held-out data. The evaluator validation guide defines the scorecard and regression testing defines CI gates.

Canary the Candidate with Explicit Stop Conditions

Set numerical values from the application’s evidence and risk appetite; the table does not prescribe universal thresholds. Start with shadow traffic, then route a small reversible segment. A canary without an owner, observation window and automatic or practiced rollback is only a smaller uncontrolled release.

Rollback Is a Data Operation as Well as a Deployment Operation

Keep the previous pinned model available in configuration and retain the question and threshold bundle that belonged to it. If a canary breaches a guardrail, revert the whole bundle. Reverting only the model while leaving a newly fitted threshold can produce a configuration that was never evaluated.

For Jev evals, preserve both versions’ results beside the same trace rather than overwriting history. For Jev policies, record which version generated the signal and which policy version converted it into an action. That provenance is necessary for incident reconstruction.

Detect Alias Movement Before It Becomes an Unexplained Metric Shift

A distribution shift after an alias moves may reflect an intended model improvement, a calibration change or a provider difference. It is still a change that needs attribution. The launch tracker supplies the public chronology; production logs remain the authoritative record of what each request used.

1. Resolve each alias on a schedule through the supported model metadata or response field.
2. Compare the observed target with the last recorded target.
3. Open a candidate evaluation when the target changes; do not update the approved bundle automatically.
4. Annotate dashboards with alias, provider and downstream model changes.
5. Retain the old result series and migration report after promotion.