Microsoft Unveils AI Agent LLM Routing Solution, Cutting Costs by Up to 85%

Microsoft has released a reference architecture for routing AI agent traffic on Azure Kubernetes Service, combining RouteLLM, agentgateway, and the Kubernetes Gateway API Inference Extension. The solution routes simple calls to cost-effective models, potentially saving up to 85% in costs, but thresholds must be calibrated based on actual traffic.