Blog
Artwork: La Berceuse by Vincent van Gogh. The Metropolitan Museum of Art · Public domain
EvaluationEvaluating a gpt-realtime Voice Agent with Simulation
How we test a speech-to-speech debriefing agent end-to-end: a persona-driven simulated technician talks to the real gpt-realtime agent over a WebSocket, and binary judges plus outcome verification grade the result.
Artwork: Saint Jerome by Jusepe de Ribera. Cleveland Museum of Art · Public domain
EvaluationSkill Evaluation with Microsoft Waza
A YAML-first, two-tier approach to evaluating AI skills: Waza validates trigger routing and output relevance at the skill layer, then implementation evals gate code quality — all in CI.
Artwork: The Crucifixion of Saint Andrew by Caravaggio. Cleveland Museum of Art · Public domain
EvaluationBeyond 1–5 Scores: Binary Judges, Outcome Verification, and pass^k
Multi-point judge scales are subjective, transcript-only scoring misses system state, and a single trial can be a lucky run. Four changes that make agent evaluation trustworthy.
Artwork: Mäda Primavesi by Gustav Klimt. The Metropolitan Museum of Art · Public domain
EvaluationHybrid Agent Evaluation: Custom Judges + Azure Foundry Evaluators
Why we combined domain-specific custom LLM judges with Azure Foundry's built-in agent evaluators over a REST API — getting behavior metrics and scenario-fact groundedness from C# without a Python dependency.
Artwork: Wheat Field with Cypresses by Vincent van Gogh. The Metropolitan Museum of Art · Public domain
EvaluationEval-Driven Development: Red-Green-Refactor for AI Agents
Every agent change should be justified by a failing evaluation. A disciplined, TDD-style process for evolving AI agents without regressions.
Artwork: Irises by Vincent van Gogh. The Metropolitan Museum of Art · Public domain
AI AgentsHybrid Activity Classification: Keyword Rules First, LLM Fallback
A two-tier classifier that answers the easy cases with deterministic keyword rules and escalates only the ambiguous ones to an LLM — cutting cost and latency while keeping a confidence threshold in control.