How to debug a failing LLM evaluation

the short answer

Start in Observe → evaluations and separate missing results from wrong results. Query the affected sessions by agent, environment and evaluation version, then run a Failproof audit to find what the failures share. Review the trace evidence, give feedback on the finding and update the evaluation. Re-evaluate individual sessions while iterating, backfill the accepted version across historical sessions, and alert if missing or incorrect scores return.

First split: Missing evaluation result or incorrect evaluation result.
Find the pattern: Queries plus a Failproof audit over affected sessions.
Fix: Update and version the evaluation from reviewed trace evidence.
Verify: Re-evaluate, backfill, monitor and alert on recurrence.

1. Decide Whether the Score Is Missing or Wrong

Open Observe → evaluations and inspect the affected agent and environment. A missing result means the evaluation did not produce a usable score for a session that should have been covered. A wrong result means the evaluation completed but its verdict does not match the trace evidence or product behavior.

Symptom	What to inspect first
No result on some sessions	Evaluation scope, agent ID, environment, deployment version and whether the required trace events exist.
Evaluation errors	The error separately from the score; do not count a failed evaluation as an agent failure.
Passes an obvious failure	The trace evidence supplied to the evaluation and the criterion for a pass.
Fails acceptable sessions	The boundary between the intended behavior and a harmless lookalike.
Scores changed suddenly	Evaluation version, model version, input shape and the agent deployment that changed at the same time.

2. Inspect Evaluation Coverage in Failproof AI

Use the dashboard or Cloud CLI to check recent results by evaluation, agent and environment. Compare the number of eligible sessions with the number of completed results. A drop in coverage points to filters, missing trace data, evaluation errors or a deployment mismatch before it points to judge quality.

fp evals --since 1h --score task_completed:0..1
fp evals --since 24h --aggregate
fp evals --agent-id checkout-agent --aggregate
fp --json evals --aggregate --env production

Open representative sessions from each group: correctly scored, incorrectly scored and missing. Check whether all three contain the messages, tool calls and tool results the evaluation needs. If the evidence never reached the trace, changing the rubric will not repair the score.

3. Query for Missing Scores and Affected Cohorts

Open Analyze → Queries and find sessions that should have an evaluation result but do not. Break the result down by session.agent_id, session.environment, time and evaluation version. Save the query once it correctly identifies the gap so it can feed a dashboard or alert.

Patterns usually become visible at this point. Missing scores may belong to one agent name, one environment, long sessions, sessions without a required event, or traffic after a particular evaluation deployment. Wrong scores may cluster around one tool, workflow or newly introduced behavior.

fp agent ask "Find production sessions from the last 24 hours that should have a task_completed evaluation but have no result. Group them by agent and environment."
fp query run <saved-query-id>

Use the Failproof agent to draft the query, verify it, then save the accepted version.

4. Run an Audit Over the Affected Sessions

Create a Failproof Cloud audit with a goal that names the symptom, such as "find why task-completion scores are missing" or "find the common behavior in sessions incorrectly marked as complete". Scope it to the affected agent, environment and time window.

The audit combines the trace evidence, available evaluation results and policy decisions across the population. Related cases become findings with analysis, severity, affected sessions, evidence queries and a recommended fix. This shows whether the defect belongs in evaluation scope, missing input evidence, the criterion itself or the agent.

fp audits list
fp audits findings --status open --limit 20

5. Review the Finding and Give Feedback

Open the sessions supporting the finding. Confirm it when the evidence is correct, dismiss it when the pattern is unsupported, or mute an expected pattern that should not keep returning. Explain the missing boundary: what evidence makes the session a pass, a failure or ineligible for this evaluation.

Failproof AI uses this reviewed pattern feedback to improve the relevant evaluation. That concentrates human judgment on recurring production behavior instead of requiring a separate spreadsheet of labels before the team can fix the problem.

6. Update and Test the Evaluation

Create a new evaluation version with the corrected scope, evidence selection or criterion. Test it first against the sessions behind the finding. Include correctly scored sessions and acceptable lookalikes as well as failures so the fix does not simply reverse the error.

Re-evaluate individual sessions while iterating. When the result matches the reviewed evidence, backfill the new version across a broader historical window. Compare coverage, evaluation errors and verdicts with the previous version before enabling it for current traffic.

Keep the previous immutable version available for rollback. A score series should show where the evaluation changed so a measurement update is not mistaken for a sudden change in agent behavior.

7. Monitor the Repaired Evaluation

Keep the saved missing-score query on a dashboard and turn it into an alert if coverage drops again. Add an evaluation_score alert for a critical result or an eval_compound alert when several evaluation conditions together identify the failure more reliably.

Continue the audit after deployment. It should show whether the original pattern disappeared and whether a new variant emerged. If the evaluation is now correct but the agent still fails, assign the finding and fix the prompt, tool, model, data or workflow.

Failproof AI has already built the evaluation execution, result history, trace evidence, missing-score queries, dashboards, audits, feedback loop, versioning, backfill and alerts needed for this process. Your team reviews the evidence and owns the product decision. You do not need to maintain a separate diagnostic dataset and scoring pipeline around every evaluation.

FAQ

Why is a Failproof AI evaluation score missing?

Check whether the session matched the evaluation scope, used the expected agent ID and environment, contained the trace evidence the evaluation needs, and completed without an evaluation error. Query missing results by cohort, then run an audit over the affected sessions to find the shared cause.

How do I fix an LLM evaluation that gives wrong scores?

Review the trace evidence behind the wrong results, use an audit to group the recurring pattern, give feedback on the finding, and create a new evaluation version with the corrected boundary. Test it on the affected sessions and acceptable lookalikes before backfilling it across history.

Can Failproof AI rerun an evaluation after I change it?

Yes. Re-evaluate individual sessions while iterating, then use backfill to run the accepted evaluation version over historical sessions. Keep versions separate so you can compare the results and roll back if needed.

How do I know the evaluation is fixed?

The sessions behind the original finding should now score correctly, acceptable lookalikes should remain acceptable, missing-result coverage should recover, and new production sessions should follow the same pattern. Keep the query, dashboard, audit and alert running to detect recurrence.

Get Started

Failproof AI is free to start. It finds recurring failure modes across agent sessions using code-based and LLM-based evaluations, groups the evidence into findings, and recommends fixes. Bring the eval suite you already have, alert the right owner when behavior drifts, and turn a tested fix into a policy that prevents the failure from recurring. See pricing for the tiers.

start free get a demo →

Sources

Products change; if a detail here is out of date, tell us at support@befailproof.ai.