Microsoft plans to deploy Maia 300 AI chips at gigawatt scale
Microsoft plans to deploy its in-house Maia 300 artificial intelligence processor into its datacentres at gigawatt scale starting next year, potentially reaching up to a million units to replace Nvidia hardware as its primary inference engine. electronicsweekly.com reported that the chiplet-based silicon relies on 2nm and 3nm nodes to power long-context reasoning, conversational agents, and multi-step tasks.
The Tech TL;DR:
- Microsoft will deploy up to one million Maia 300 AI processors at gigawatt scale starting next year.
- Each rack packs 96 units into a 167-kilowatt envelope, yielding 40% more tokens per megawatt than top GPU systems.
- The hardware is engineered to overtake Nvidia processors for inference tasks within Microsoft datacentres.
Deploying Maia 300 at Gigawatt Scale
According to Rani Borkar at Microsoft, the company feels confident in having built the best product for these specific workloads and is backing that confidence by deploying Maia at gigawatt scale starting next year. Each rack holds 96 Maia 300 chips within a 167-kilowatt power envelope. This thermal and electrical density delivers 40% more tokens per megawatt than top graphics processing unit systems while achieving up to seven times the rack-scale throughput of the previous-generation Maia 200.
Transitioning From Maia 200 to In-House Silicon
The hardware transition follows the earlier deployment of the Maia 200 chip, which began appearing in Microsoft datacentres earlier this year. That predecessor currently runs workloads for Copilot and Teams. While Microsoft started later than competitors Amazon and Google in getting proprietary chips into production servers, the Maia 300 is planned to supersede Nvidia processors as the most heavily utilized AI inference processor across the entire Microsoft datacentre portfolio.
Architectural Efficiency for Agentic Workloads
Built using an advanced mix of 2nm and 3nm chiplets, the Maia 300 architecture specifically targets long-context reasoning and multi-step conversational AI agents.
# Example Kubernetes resource configuration for an AI inference cluster
apiVersion: apps/v1
kind: Deployment
metadata:
name: maia-inference-workload
spec:
replicas: 96
selector:
matchLabels:
app: conversational-agent
template:
metadata:
labels:
app: conversational-agent
spec:
containers:
- name: tensor-engine
image: mcr.microsoft.com/ai/inference:maia-300-latest
resources:
limits:
accelerator.microsoft.com/maia: "1"
memory: "512Gi"
requests:
accelerator.microsoft.com/maia: "1"
memory: "256Gi"
Disclaimer: The technical analyses and security protocols detailed in this article are for informational purposes only. Always consult with certified IT and cybersecurity professionals before altering enterprise networks or handling sensitive data.