How to score task completion for AI agents

Why "Did It Finish" Is Harder than It Sounds

Agents report their own success, and they are optimistic reporters. The last message of a session is usually some version of "all done", whether or not anything is. The session ending is not evidence either: sessions end because the task finished, because a step budget ran out, because the model decided to stop, or because the user closed the tab. A completion metric that reads the last message or counts ended sessions measures confidence, not completion.

In practice each run lands in one of five places, and a useful metric keeps them apart:

• Complete. Every condition of done is met, by evidence.
• Correctly declined or escalated. The agent could not or should not finish, and handed off with a reason. For many agents this is a success.
• Partial. Some conditions met. The refund went through; the ticket was never updated.
• Claimed but not done. The agent said it finished and it did not. The most expensive category, because nobody goes back to check.
• Not done, and says so. An honest failure. Cheap to catch, because the agent told you.

Write Down "Done" for Each Agent First

Before any code, write one "done when" statement per agent in terms of things you can observe after the run. "Helped the customer" is not observable. "The customer has a refund confirmation or an escalation note, and the ticket is resolved or escalated" is. Each clause should map to a check, even if some checks end up being a judge reading the transcript.

Writing these down surfaces disagreements early. If two engineers write different done-when lines for the same agent, you have found a spec problem, and no metric would have fixed it.

Test each definition before automating it. Take ten recent sessions, apply the done-when line by hand, and write down every case where you hesitated. Each hesitation is a missing clause: does a refund for the wrong amount count? Does a PR that is open but failing CI count? Settle those in the text now, because a judge given an ambiguous definition will settle them for you, differently each time.

Check the End State, Not the Transcript

The most reliable completion check ignores the conversation and inspects the world afterwards. τ-bench, a 2024 benchmark for tool-using agents, scores a run by comparing the database state at the end of the conversation with an annotated goal state (Yao et al., 2024). The same idea works on your own systems: query the thing the agent was supposed to change.

import subprocess


def coding_task_done(repo: str, base_sha: str, test_cmd: list[str]) -> dict:
    def run(*cmd: str) -> subprocess.CompletedProcess:
        return subprocess.run(cmd, cwd=repo, capture_output=True, text=True)

    new_commits = run("git", "rev-list", "--count", f"{base_sha}..HEAD").stdout.strip()
    checks = {
        "tests_pass": run(*test_cmd).returncode == 0,
        "tree_clean": run("git", "status", "--porcelain").stdout.strip() == "",
        "committed": int(new_commits or 0) > 0,
    }
    checks["done"] = all(checks.values())
    return checks


def state_diff(expected: dict, actual: dict) -> list[str]:
    """Compare the end state with the goal state, field by field."""
    return [
        f"{key}: expected {want!r}, got {actual.get(key)!r}"
        for key, want in expected.items()
        if actual.get(key) != want
    ]


print(coding_task_done("/work/checkout-service", "3f9c2e1", ["pytest", "-q"]))
# for example: {'tests_pass': True, 'tree_clean': True, 'committed': False, 'done': False}
print(state_diff({"refund_amount": 30.0, "ticket_status": "resolved"},
                 {"refund_amount": 30.0, "ticket_status": "open"}))
# ["ticket_status: expected 'resolved', got 'open'"]

Record the base commit (or a snapshot of the relevant records) before the run starts; an end-state check needs a "before" to compare with. For offline evals, reset the environment between runs so one task's leftovers do not complete the next one.

"Done" also means nothing else happened. A refund issued twice passes a check that looks for a refund; a coding agent that fixed the bug and also rewrote an unrelated module passes a test run. Where it is cheap, check the negative too: exactly one refund row, a diff limited to the files the task named, no new records in tables the agent had no reason to touch. These checks are where completion scoring and safety scoring meet.

A Judge When There Is No State to Check

Some tasks leave nothing to query: a research summary, an answer to a question, advice. Then a judge reads the transcript, but it applies the same done-when definition, and it is told to ignore how confident the agent sounds. Ask for a status and, separately, whether the agent claimed to be finished. The combination of "claimed" and "not complete" is the number to watch.

# completion_judge.py
import json
import os

import anthropic

JUDGE_MODEL = os.environ["JUDGE_MODEL"]
client = anthropic.Anthropic()                    # reads ANTHROPIC_API_KEY
STATUSES = ("complete", "partial", "not_complete")

RUBRIC = """You decide whether an AI agent finished its task.
The DEFINITION OF DONE is the only standard. Ignore how confident the agent sounds.
status: "complete" if every condition in the definition is met by evidence in the transcript,
"partial" if some are, "not_complete" if none are.
claimed_done: true if the agent told the user or the system that the task was finished.
Reply with JSON only, for example:
{"status": "partial", "claimed_done": true, "evidence": "refund issued at step 12; ticket never updated"}"""


def parse_json(text: str) -> dict | None:
    start, end = text.find("{"), text.rfind("}")
    if start == -1 or end <= start:
        return None
    try:
        return json.loads(text[start:end + 1])
    except json.JSONDecodeError:
        return None


def judge_completion(done_when: str, transcript: str) -> dict | None:
    prompt = f"{RUBRIC}\n\nDEFINITION OF DONE:\n{done_when}\n\nTRANSCRIPT:\n{transcript}"
    msg = client.messages.create(
        model=JUDGE_MODEL, max_tokens=512,
        messages=[{"role": "user", "content": prompt}],
    )
    text = msg.content[0].text
    verdict = parse_json(text)
    if verdict is None or verdict.get("status") not in STATUSES or not isinstance(verdict.get("claimed_done"), bool):
        return None   # not scored; count these separately
    return verdict

Calibrate this judge against human labels before it feeds a dashboard. Completion judges tend to be lenient in a specific way: they accept the agent's own summary as evidence. The rubric's "ignore how confident the agent sounds" line helps, and so does giving the judge the tool results rather than only the messages. Calibrating a judge has the agreement code.

Partial Credit, and Why It Bites

It is tempting to score a run 0.5 when it did half the job. The trouble starts when you average. A completion rate of 60% could mean six in ten runs finished, or every run got 60% of the way. Those are different products with different fixes, and the average cannot tell you which one you have. Checklist-based partial credit adds a second problem: every item weighs the same, so an agent that reliably does the three easy steps and never the hard one scores 75% on a task nobody would call three-quarters done.

Keep them apart. The headline number is binary: complete (or correctly escalated) versus not. Progress through the steps is a diagnostic you read when completion drops, to see where runs stall. AgentBoard makes the same split for multi-turn agents, reporting a progress rate beside success rate because success alone reveals little about how far failing agents got (Ma et al., 2024).

And report the binary rate with its uncertainty. 42 completions out of 50 is 84%, with a 95% Wilson interval of 71.5% to 91.7%; how many eval examples you need shows the calculation and when 50 is enough.

Measuring Task Completion with Failproof AI

In Failproof AI, start with an agent contract that states the agent's purpose, expected outputs, definition of done and prohibited behavior. Use the same definition in the completion evaluation so individual scores and broader failure analysis measure success against one standard.

Use a code-based evaluation when completion can be verified from an end state, and an LLM-based evaluation when success requires interpreting the session. You can also bring the evaluation suite you already have. Keep completion, partial progress, correct escalation and false claims of completion as separate results so the headline rate stays meaningful and each failure points toward a different fix.

Each evaluation result stays linked to the trace evidence behind it. That makes "claimed done but not complete" especially useful: a reviewer can see the agent's closing claim beside the missing tool result, unchanged record or unfinished step instead of trusting a score without context.

Alert when completion declines or false completion claims increase. Across many sessions, Failproof AI combines evaluation results and trace evidence to find recurring failure modes, groups affected sessions into findings, and recommends what to change. That can distinguish, for example, an agent that routinely skips the final ticket update from one that cannot complete the refund itself.

Fix the underlying prompt, tool or workflow based on that evidence. For a narrow requirement that can be checked when an agent tries to finish - such as requiring a test run before a coding task is declared complete - a tested behavioral policy can provide immediate feedback. Keep the completion evaluation running afterward to confirm the behavior improves rather than merely changing the closing message.

When You Do Not Need a Completion Score

If the task has a natural pass/fail signal you already collect - a CI run, a payment that settles, a form that validates - that signal is your completion metric, and a separate eval adds cost without adding information. The same goes for agents a human reviews every time: the reviewer's accept or reject is a better label than any judge. Add a completion score when nobody looks at most runs, which is to say, as soon as the agent is useful.

You also do not need a judge where a check exists. If an end-state check covers an agent, run it on every session and keep the judge for the sessions it cannot reach. Paying a model to guess what a database query would tell you is the most avoidable cost in completion scoring.