#llm-judge
Artwork: Wheat Field with Cypresses by Vincent van Gogh. The Metropolitan Museum of Art · Public domain
EvaluationDeepEval: Pytest-Native Evaluation for LLM Applications and Agents
DeepEval is the strongest general-purpose starting point for Python LLM, RAG, and agent evaluation. Start with deterministic outcomes, then add calibrated semantic metrics where code falls short.
Artwork: The Crucifixion of Saint Andrew by Caravaggio. Cleveland Museum of Art · Public domain
EvaluationBeyond 1–5 Scores: Binary Judges, Outcome Verification, and pass^k
Multi-point judge scales are subjective, transcript-only scoring misses system state, and a single trial can be a lucky run. Four changes that make agent evaluation trustworthy.
Artwork: Mäda Primavesi by Gustav Klimt. The Metropolitan Museum of Art · Public domain
EvaluationHybrid Agent Evaluation: Custom Judges + Azure Foundry Evaluators
Custom judges test scenario facts while Azure Foundry evaluators score general agent behavior. A REST boundary gives C# both views without adding Python to the application.
Artwork: Wheat Field with Cypresses by Vincent van Gogh. The Metropolitan Museum of Art · Public domain
EvaluationEval-Driven Development: Red-Green-Refactor for AI Agents
Every agent change should be justified by a failing evaluation. A disciplined, TDD-style process for evolving AI agents without regressions.