multiple jev questions in one call

Batch Measurements, Not Decision Dependencies

A support message can be evaluated for queue, urgency, refund request and abusive language in one call because every question reads the original ticket. The results do not need one another. This avoids repeating state across separate calls and creates a single transport boundary.

A second decision such as “which billing specialist should handle the ticket?” depends on the first queue result and a new candidate list. Put that result and candidates into a second request. Explicit orchestration makes the dependency testable and prevents readers from assuming Jev performs an undocumented reasoning chain.

Draw the Question Graph Before Choosing Call Boundaries

Name Questions for Software and Define Meaning in Prose

Question IDs map response fields back to application code. TypeSafe says IDs are not part of the semantic instruction, so an ID such as urgent should not replace an explicit definition. Put the criterion in instructions and, for Choice or Score, define every option or level. Version those definitions independently from the identifier consumed by code.

Avoid contradictory questions in the same request. TypeSafe’s Jev 1.13 notes warn that contradictions and apparently equivalent positive/negative formulations can behave unexpectedly. Ask one canonical proposition and derive exact complements in code.

Worked Graph: Ticket Triage and Specialist Selection

The first three judgments can share a call because each reads the same evidence and has a meaning independent of the others. The specialist decision cannot: its candidate set exists only after the route is known. Refund eligibility is exact business logic and should not be represented as another semantic question. This graph prevents a batch from becoming an implicit reasoning chain.

Batching Changes Tokens, Latency and Failure Scope

Shared state is serialized once, while every question adds its own instructions and criteria. Batching can lower repeated input cost and network overhead, but an oversized batch approaches total-request and longest-question limits and may make retrying one failed transport repeat every judgment. Measure actual serialized tokens and p50/p95/p99 latency across representative batch sizes.

Do not infer linear speedup from the word “parallel.” TypeSafe’s architecture claim describes model behavior, not a guarantee that end-to-end latency stays constant as questions grow. Provider queuing, rate limits, payload size and SDK retry behavior remain part of the observed system.

Budget State Once and Every Question Explicitly

Estimate batch cost from the serialized request, not from question count alone. One long shared transcript with two short questions can cost more than many questions over compact state. Conversely, splitting independent questions repeats state and network overhead. The pricing guide provides token calculations, while state design covers projection and provenance.

A Transport Success Is Not per-Question Validity

Validate that every expected question ID exists, its answer matches the declared primitive and its distribution is finite and complete. A successful HTTP response should not cause missing or malformed answers to become false, zero or approval. Record a distinct invalid state and send it through the same fallback path as other unavailable evidence.

If one result is invalid, decide whether the application can safely consume the other independent answers or must reject the batch atomically. That is an application contract, not a documented Jev guarantee. Keep it explicit and test it with synthetic malformed responses. The HTTP response guide and retry guide cover validation and failure handling.

Preserve per-Question Identity Inside a Batched Call

A batch is a transport optimization, not the unit of model quality. One question can regress while aggregate request success stays green. Connect every result to its own evaluation criterion and calibration.

• Log one request ID and one stable question version per answer.
• Store each full typed distribution and the derived action separately.
• Attribute labels and downstream outcomes to the relevant question, not merely the batch.
• Track batch size, state tokens, question tokens, latency, provider errors and retries.
• Replay both individual questions and representative batches after a version change.

Benchmark Batch Size as a System Parameter

Do not claim that parallel model evaluation makes a batch N times faster. The relevant comparison is end-to-end work completed per unit time at an acceptable error and failure rate. Preserve raw artifacts so future model or provider changes can rerun the same matrix.

1. Freeze representative state documents and independent question sets of several sizes.
2. Run single-question requests and equivalent batches through the same pinned provider and model.
3. Compare answer distributions to detect quality changes rather than checking only selected labels.
4. Measure serialized input tokens, total cost, p50/p95/p99 latency, throughput, 429s and retries.
5. Repeat with short, typical and near-limit state and with realistic concurrency.
6. Choose a maximum batch size and split strategy from the observed latency and failure budget.