Technical writing on software engineering, systems, and applied AI.https://blog.ofdeme.com/enhttps://blog.ofdeme.com/blog/simulation-eval-gpt-realtime-voice-agent/https://blog.ofdeme.com/blog/simulation-eval-gpt-realtime-voice-agent/How we test a speech-to-speech debriefing agent end-to-end: a persona-driven simulated technician talks to the real gpt-realtime agent over a WebSocket, and binary judges plus outcome verification grade the result.Mon, 17 Aug 2026 00:00:00 GMTEvaluationevaluationvoicegpt-realtimesimulationazureOmer Demirhttps://blog.ofdeme.com/blog/waza-skill-evaluation/https://blog.ofdeme.com/blog/waza-skill-evaluation/A YAML-first, two-tier approach to evaluating AI skills: Waza validates trigger routing and output relevance at the skill layer, then implementation evals gate code quality — all in CI.Tue, 05 May 2026 00:00:00 GMTEvaluationevaluationwazaskillsciai-agentsOmer Demirhttps://blog.ofdeme.com/blog/binary-judges-outcome-scoring-pass-k/https://blog.ofdeme.com/blog/binary-judges-outcome-scoring-pass-k/Multi-point judge scales are subjective, transcript-only scoring misses system state, and a single trial can be a lucky run. Four changes that make agent evaluation trustworthy.Tue, 21 Apr 2026 00:00:00 GMTEvaluationevaluationllm-judgeai-agentsreliabilitycalibrationOmer Demirhttps://blog.ofdeme.com/blog/hybrid-agent-evaluation-custom-judges-azure-foundry/https://blog.ofdeme.com/blog/hybrid-agent-evaluation-custom-judges-azure-foundry/Why we combined domain-specific custom LLM judges with Azure Foundry's built-in agent evaluators over a REST API — getting behavior metrics and scenario-fact groundedness from C# without a Python dependency.Tue, 14 Apr 2026 00:00:00 GMTEvaluationevaluationazure-foundryllm-judgeai-agentsdotnetOmer Demirhttps://blog.ofdeme.com/blog/eval-driven-development-for-ai-agents/https://blog.ofdeme.com/blog/eval-driven-development-for-ai-agents/Every agent change should be justified by a failing evaluation. A disciplined, TDD-style process for evolving AI agents without regressions.Fri, 10 Apr 2026 00:00:00 GMTEvaluationevaluationai-agentstestingllm-judgeprocessOmer Demirhttps://blog.ofdeme.com/blog/hybrid-activity-classification-keyword-llm/https://blog.ofdeme.com/blog/hybrid-activity-classification-keyword-llm/A two-tier classifier that answers the easy cases with deterministic keyword rules and escalates only the ambiguous ones to an LLM — cutting cost and latency while keeping a confidence threshold in control.Thu, 02 Apr 2026 00:00:00 GMTAI Agentsclassificationllmazure-openaicost-optimizationmulti-agentOmer Demirhttps://blog.ofdeme.com/blog/real-time-voice-debriefing-voice-live-api/https://blog.ofdeme.com/blog/real-time-voice-debriefing-voice-live-api/How the field service assistant runs a hands-free, 8-step voice debrief over a persistent WebSocket loop — and how persona, scenario, and environment fixtures make it testable.Tue, 24 Mar 2026 00:00:00 GMTAI Agentsvoiceazurerealtimemulti-agenttestingOmer Demirhttps://blog.ofdeme.com/blog/multi-agent-field-service-assistant-architecture/https://blog.ofdeme.com/blog/multi-agent-field-service-assistant-architecture/The architecture of a field service assistant built with FastAPI and Pydantic v2: specialized briefing, debriefing, and classification agents behind an orchestrator, with swappable connectors and persistence.Tue, 10 Mar 2026 00:00:00 GMTAI Agentsmulti-agentfastapipydanticazure-openaiarchitectureOmer Demirhttps://blog.ofdeme.com/blog/chunking-strategies-structured-documents/https://blog.ofdeme.com/blog/chunking-strategies-structured-documents/A pluggable chunker design — simple, markdown, section-aware, and semantic — for splitting structured documents before vector indexing, with metadata that survives retrieval.Thu, 18 Sep 2025 00:00:00 GMTRAGragchunkingembeddingsretrievalpythonOmer Demirhttps://blog.ofdeme.com/blog/slm-rag-langchain-onnx-runtime-genai/https://blog.ofdeme.com/blog/slm-rag-langchain-onnx-runtime-genai/Part two: a custom LangChain LLM wrapper over ONNX Runtime GenAI, streaming token generation, and building an optimized Phi-3 ONNX graph for offline Windows RAG.Fri, 13 Jun 2025 00:00:00 GMTMachine LearningslmonnxraglangchainwindowsOmer Demirhttps://blog.ofdeme.com/blog/slm-onnx-runtime-genai-windows-benchmark/https://blog.ofdeme.com/blog/slm-onnx-runtime-genai-windows-benchmark/How we benchmarked ONNX Runtime GenAI, LlamaCPP, Hugging Face Optimum, and Triton to run a sub-5-second RAG pipeline offline on Windows.Fri, 30 May 2025 00:00:00 GMTMachine LearningslmonnxraginferencewindowsOmer Demirhttps://blog.ofdeme.com/blog/managed-identity-logic-app-consumption-terraform/https://blog.ofdeme.com/blog/managed-identity-logic-app-consumption-terraform/Terraform does not natively wire Managed Identity API connections for Logic App Consumption. Here is how azapi_resource plus an ARM template deployment closes the gap.Fri, 31 Jan 2025 00:00:00 GMTAzureterraformlogic-appsmanaged-identityazureiacOmer Demir