Microsoft unveils Maia 300, an inference-first chip

What happened: Microsoft detailed Azure Maia 300, its third-generation accelerator built for inference, and said it will scale the Maia family to multi-gigawatt deployments starting in 2027, making Maia the primary inference system for Microsoft’s products and services. A Maia 300 rack puts 96 accelerators in a 167-kilowatt envelope with 27.6 TB of HBM and 1.7 PB/s of aggregate memory bandwidth. Each chip carries eight HBM4 stacks delivering 18.0 TB/s of peak bandwidth within a 950-watt TDP, and the compute chiplets are built on TSMC’s N2P node.

Context: Microsoft expects Maia 300 to deliver 40% more tokens per megawatt than leading GPU-based systems and up to 7x the rack-scale token throughput of Maia 200. Maia 200, the previous generation, is now serving production workloads including Copilot, with GitHub Copilot and Microsoft Foundry to follow by the end of the year, running OpenAI models, Microsoft’s own MAI models and leading open-weight families. Production deployments of Maia 300 begin in 2027.

Why it matters: A hyperscaler designing its own silicon for inference is a direct bet on cost and control against merchant GPUs, and the tokens-per-megawatt figure is the number to watch. It also shows Microsoft co-designing chips and software around the specific workloads it runs at scale, from latency-sensitive chat to long-running agentic tasks.

(Original source: Microsoft)