Chapter 13
A high score can hide the failures that matter
Updated
A model change improves average success, but users report more incorrect actions.
Inspect
Slice by task, customer, severity, and workflow version. Inspect resulting system state as well as the answer. Validate the grader separately on expert-labeled pass and fail cases.
Decide
Keep a representative estimate of normal performance alongside a deliberately difficult regression suite. Track false acceptance of harmful outcomes separately; do not bury it in an average.
Bring to a review
A release table: baseline → candidate, task slice, sample size, uncertainty, severe failures, latency, and decision owner.
Common trap: Repeatedly tuning against the held-out set quietly turns it into development data.
Read
- AI Evals: Everything You Need to Know Error discovery, representative sampling, and validated automated judges.Focus: Start with error analysis, sampling traces, stale datasets, and validating an LLM judge against expert labels. Apply: Create a labeled failure taxonomy and report judge false accepts and false rejects on held-out examples.
- Demystifying evals for AI agents Task, trial, grader, trace, and outcome: a vocabulary for evidence-driven delivery.Focus: Read tasks, trials, graders, transcripts, outcomes, and repeated-run metrics. Apply: Build a release report with outcome checks, multiple trials where appropriate, and unresolved failure slices.