Microsoft's Maia 300 Makes Its Own Silicon the Default for AI Inference

Microsoft introduced the third generation of its in-house AI accelerator on 12 October 2026, and paired it with a claim about how it intends to run AI from next year onward. The Azure Maia 300, unveiled at the OCP Global Summit in San Jose, will enter production deployments in 2027, and Microsoft says the Maia family will scale to multi-gigawatt deployments from then on, making it the primary inference system across the company's products and services.

Related reading: SpaceX in Talks to Raise $40 Billion to Buy Nvidia AI Chips · AMD Will Buy World Labs for $8.2 Billion and Make Fei-Fei Li Its Chief Scientist.

The news came in a blog post by Scott Guthrie, Microsoft's executive vice president for Cloud + AI, on the official Microsoft blog. A separate Azure technical deep dive carried the architecture details. Rani Borkar, president of Azure hardware systems and infrastructure, gave the interview quotes that accompanied the launch.

A chip built for running models, not training them

Inference is the work of putting a trained model to use: answering prompts, generating text, holding a conversation, working through a chain of steps. It happens every time a person or an agent touches an AI product, which makes it the recurring cost once a model is in service. Microsoft has designed the Maia family around that half of the problem from its first generation, and the Maia 300 is the most direct expression of it so far.

Microsoft's stated specifications put 288GB of HBM4 memory on each chip, spread across eight HBM4 stacks, with 18.0 TB/s of peak memory bandwidth and a 950-watt thermal design point. Microsoft lists up to 13.8 PFLOPS of throughput in the FP4 format. The chip is built from several chiplets rather than one slab of silicon: the compute chiplets use TSMC's N2P 2-nanometre process, while the parts that move data use a cheaper 3-nanometre node. Splitting the design that way puts the expensive transistor where it changes the result and uses older silicon for memory control and input and output.

At rack level, Microsoft places 96 Maia 300 accelerators in a system that draws about 167 kilowatts and carries 27.6TB of HBM, with 1.7 PB/s of aggregate bandwidth across the scale-up domain. The Next Web, citing Reuters, noted for comparison that Nvidia's most recent GB300 rack holds 72 GPUs at about 142 kilowatts. The Maia networking runs over Ethernet instead of a proprietary fabric, and Microsoft says the topology scales from a single rack to a whole data centre.

The performance figures, and the gap inside them

Microsoft says the Maia 300 delivers about 40% more tokens per megawatt than "leading GPU-based systems," and up to seven times the rack-scale token throughput of its predecessor, Maia 200, in a full 96-accelerator configuration. Tokens per megawatt is a power-efficiency measure: it describes how much model output a system produces for each unit of electricity consumed. Electricity is a running cost that does not stop.

"We expect Maia 300 to deliver 40% more tokens per megawatt compared to leading GPU-based systems. And in a full 96-accelerator configuration, it delivers up to 7x the rack-scale token throughput of Maia 200." — Scott Guthrie, executive vice president for Cloud + AI, Microsoft

The comparison behind the 40% figure is not disclosed. Microsoft did not name the systems it measured against, and no independent benchmark of the Maia 300 has been published. The seven-times figure measures the new chip against Microsoft's own previous generation. It describes the pace of the internal roadmap, not Microsoft's position next to Nvidia.

One figure in circulation belongs to the wrong chip. The 1.7-times-more-tokens-per-watt number that has been attached to the Maia 300 is a Maia 200 result, and Microsoft has cited it for its own built-in AI features, Teams, Copilot and Cowork. The Maia 200 is the generation actually in service.

Maia 200 is the one already doing the work

The Maia 200, introduced on 26 January 2026, is the generation Microsoft can point to in production. It uses TSMC's 3-nanometre process, carries 216GB of HBM3e at 7 TB/s, holds more than 140 billion transistors and is rated at 750 watts, with more than 10 PFLOPS of FP4 throughput. Microsoft says it runs production workloads for Microsoft Copilot, for the company's MAI models and for OpenAI models on Azure, with GitHub Copilot and Microsoft Foundry to follow by the end of 2026. It is deployed in the United States, Europe and Australia.

At its own launch, Microsoft claimed the Maia 200 had three times the FP4 performance of Amazon's third-generation Trainium and FP8 performance above Google's seventh-generation TPU. That was Microsoft's comparison, made by the company about its own part. It is still the clearest statement of what the Maia family is for: an internal supply that shifts Microsoft's own demand away from the merchant market, not a merchant chip sold to outside customers.

Why the inference half is the expensive one to own

Training a frontier model is capital-intensive and episodic. A lab buys a large cluster, runs a training job for weeks or months, and the result is a model. Inference runs for as long as the product is live, and it scales with users, not with research plans. Every Copilot query, every agent step, every summarised email is an inference request that a data centre has to answer at a cost low enough to give the feature away or bundle it.

That is why Microsoft frames the Maia 300 around tokens per megawatt, not peak throughput alone. A rack drawing 167 kilowatts costs money every hour it runs, and Microsoft expects to spend more than $175 billion on capital expenditure this calendar year. Shaving 40% off the power needed for a given amount of output, if the claim holds, is a saving that compounds across a fleet.

The "primary inference system" language is the strategic part. It implies that most of the AI traffic Microsoft serves will eventually run on its own silicon instead of Nvidia GPUs. CNBC reported that Alistair Speirs, a general manager at Microsoft, declined to give a timeline for reaching multi-gigawatt scale. "We reserve a little bit of optionality there, because this is an industry that's changing very fast, and we want to deploy the right silicon for the right models that we're using at the right time as well," Speirs said.

What it changes in the race with Nvidia

Nvidia's lead rests on two things: the hardware and the software ecosystem built around it. A cloud provider that designs its own accelerator avoids paying a supplier's margin, and it can tune the chip to the specific models it serves. The trade is that in-house silicon usually trails the best merchant parts on raw performance and takes years to reach volume.

Microsoft does not have to beat Nvidia on every workload for the strategy to pay. It only has to run Microsoft's own models well enough, on the highest-volume workloads, to move them off the merchant market. Microsoft Copilot, Teams and the MAI models are the kind of high-volume, cost-sensitive traffic where an efficient custom part makes sense, and where Microsoft controls the model, the serving stack and the workload mix.

The wider context is a wave of custom silicon. Google and Amazon have shipped several generations of their own accelerators, and AI labs including OpenAI have pursued their own designs, all with the same aim of loosening Nvidia's grip on the market. Microsoft has been the laggard among the big cloud providers on this front, which is part of why the scale of the Maia 300 plan reads as a catch-up as much as a lead.

Reuters reported in August that Microsoft planned to unveil the Maia 300 this autumn. At launch, Reuters put the deployment plan at potentially more than one million Maia 300 chips next year. Every one of those chips depends on TSMC's N2P line, the same scarce leading-edge capacity that every other AI chip designer is queuing for.

Caveats

Three things are worth separating from the announcement. The first is timing: production deployments begin in 2027, so nothing here describes hardware running today. The second is scale. More than a million chips in 2027 is a plan, not a shipped figure, and Microsoft has not committed to it publicly in those terms. The third is the comparison baseline. The headline efficiency claim is against unnamed GPU systems, and the only named comparison is with Microsoft's own previous generation.

There is also the question of coverage. Microsoft's product surface is broad, and the Maia family has to run everything from small, latency-sensitive chat models to long-context reasoning and multi-step agent tasks if it is to be the primary inference system. Microsoft says the Maia 300 is optimized for long-context reasoning, latency-sensitive conversation and agentic work, with enhanced low-precision MXFP4, on-die SRAM used as a software-managed scratchpad and a custom AI transport over Ethernet. Whether the chip is as broad in practice as that list suggests will not be known until the workloads are live.

What to watch

Microsoft has said it will share more detail at its Ignite 2026 conference in November, and a Maia 300 technical paper is already available. Independent benchmarks would be the first real test of the efficiency claims.

The second test is the ramp. Executing a million-chip deployment in a single year demands supply, packaging, power and data centre capacity to arrive together, and Microsoft has kept its timeline deliberately flexible. A slip would leave the company leaning on bought accelerators for longer than it has said.

The third is whether Nvidia's software advantage still outweighs the cost of owning the silicon. Microsoft controls its models and its cloud, which is the best position from which to make custom silicon pay. It is also the position from which Google and Amazon have been working for years, and neither has dislodged Nvidia. Microsoft's bet is that inference, where the workload is stable and the volume is enormous, is the right place to start.

Cover image: "Microsoft Data Center Middenmeer, the Netherlands - 1.jpg" by Hay Kranen, CC BY 4.0, via Wikimedia Commons. A Microsoft data centre in the Netherlands.