#reliability
Artwork: The Crucifixion of Saint Andrew by Caravaggio. Cleveland Museum of Art · Public domain
EvaluationBeyond 1–5 Scores: Binary Judges, Outcome Verification, and pass^k
Multi-point judge scales are subjective, transcript-only scoring misses system state, and a single trial can be a lucky run. Four changes that make agent evaluation trustworthy.