Nvidia's Vera Rubin NVL72 Debuts in MLPerf With a 3.7x Inference Claim

On September 16, 2026, MLCommons published MLPerf Inference v6.1, the twice-yearly audit of how fast servers turn inputs into model outputs. The same release handed Nvidia a stage for the hardware it wants to sell next: a first preview submission for Vera Rubin NVL72, the rack-scale system that follows Blackwell and GB300. Nvidia says the platform reached up to 3.7x the throughput of GB300 NVL72 on the Qwen3-VL vision-language benchmark and up to 2.5x on DeepSeek-R1.

Related reading: Nvidia's DSX POD Turns an AI Factory Into a Shipping Container · Nvidia's China AI-Chip Share Falls to 34 Percent as Huawei Closes the Gap.

The numbers carry a qualifier. In MLPerf, "preview" is an availability category for hardware that need not be shipping when results are submitted, only generally available by the following round. Both of Nvidia's headline comparisons are against its own previous generation, not a rival's current part, and the company describes them as early results. They were reviewed by MLCommons, but they are not the same as a production submission.

What MLPerf Inference measures

The suite exists to make inference results comparable. In the Closed division, submitters must run the same model as the reference implementation and meet the same accuracy constraints, so differences in throughput reflect hardware and software rather than a modified model. Datacenter systems are tested in offline, server and interactive scenarios. Offline measures raw aggregate throughput with no latency bound, server adds a Poisson arrival rate and a latency target, and interactive sets a tighter per-token response requirement.

This round drew a record 30 submitting organizations and 486 datacenter and edge results, according to MLCommons. It added an End-to-End Retrieval-Augmented Generation pipeline and an Edge Agentic test modelled on coding assistants, and it produced first peer-reviewed figures for Nvidia's Vera Rubin NVL72, AMD's Instinct MI350P, Intel's Arc Pro B70 and AMD's Ryzen AI Max+ 395 (StorageReview). MLCommons said the best per-accelerator DeepSeek-R1 server result was 5.7x better than a year earlier, a reminder that the benchmark moves with the industry rather than standing still.

The suite is also changing. This round let submitters use an API-centric harness that runs a true client/server test against a live endpoint, and 16 of the 30 did so. That harness is the foundation of MLPerf Endpoints, which opens on-demand rolling submissions in October 2026 and is set to replace MLPerf Inference as the datacenter benchmark in 2027. The direction of travel is toward measuring the pipelines operators actually deploy, including multi-step and agentic workloads, rather than single model calls.

Nvidia's first Vera Rubin numbers

Nvidia submitted Vera Rubin results on two workloads, DeepSeek-R1 and Qwen3-VL. On Qwen3-VL it used vLLM with Dynamo, Nvidia's open-source inference framework; on DeepSeek-R1 it used its TensorRT-LLM library. The 3.7x figure is the peak across scenarios, not a uniform gain: Nvidia's own chart shows Qwen3-VL at about 1.8x GB300 in the offline scenario, 1.9x in server and 3.7x in the interactive scenario, as reproduced by Wccftech. The headline number is the interactive case, which is the lightest load and the one most sensitive to per-token latency.

Nvidia attributes the delta to design choices that matter for mixture-of-experts models. The company says Vera Rubin improves both the prefill and decode stages through enhanced Tensor Cores and a Transformer Engine, uses NVFP4 precision to cut the memory footprint of weights, attention and KV cache, and leans on disaggregated serving that splits prefill from decode with large-scale expert parallelism. The rack's scale-up domain runs on sixth-generation NVLink and NVLink Switch, which Nvidia says delivers 10x higher packet rates and 3x lower latency than standard Ethernet. The interconnect matters here because expert parallelism moves a large volume of tokens between GPUs, and a slow fabric would erase the gains from faster compute.

Nvidia was not the only submitter on the hardware. Nebius ran DeepSeek-R1 on 36 Rubin GPUs across nine nodes and reported 16,427 tokens per second per GPU in the server scenario, which it called the leading per-GPU server result. Because preview submissions ran at different cluster sizes, per-GPU throughput is the fairest comparison, and Nebius notes its result carried the same preview label. Its full 72-GPU GB300 NVL72 rack, by contrast, was a production submission and ranked first in DeepSeek-R1 server and offline, at 603,023 and 689,961 tokens per second.

Nvidia also pointed beyond the formal results. It says Vera Rubin NVL72 delivered 30x better performance than GB300 NVL72 on SemiAnalysis AgentX, an agentic test, in preview testing. That figure is not part of the MLPerf submission and is not comparable to the reviewed results. Nvidia expects the upcoming MLPerf Endpoints benchmark to bring standardised measurement to agentic inference, which is the kind of workload its 30x claim is aimed at.

Scaling and software as the rest of the pitch

Nvidia frames inference economics around three levers: throughput per rack, scaling efficiency as racks are added, and software improvement over time. The scaling claim is the most concrete. A DeepSeek-R1 submission scaled from one GB300 NVL72 rack of 72 GPUs to four racks of 288 GPUs and held 99% scaling efficiency in the offline scenario. The underlying figures, 679,740 tokens per second on one rack and 2,705,130 on four, show throughput rising almost in proportion to the hardware added. That matters because GPUs are the expensive part: if doubling the rack count delivered only a small throughput gain, the second rack would not pay for itself.

The software lever is real but harder for buyers to see. Nvidia says GB300 NVL72 throughput on Qwen3-VL rose up to 1.6x over its v6.0 result through lower KV-cache precision, extra kernel fusion and disaggregated serving with vLLM and Dynamo. It also published post-submission figures on GPT-OSS-120B and DLRMv3 that it says improved further, though those have not been verified by MLCommons and should be treated as vendor claims. The pattern across the round, seen from AMD and Intel as well, is that software is still moving benchmark scores on unchanged silicon.

A next-generation platform in a tight memory market

Vera Rubin NVL72 pairs the Rubin GPU with Nvidia's Vera CPU, which began shipping at scale in late August. Nvidia calls Vera its "first CPU built for agents", with 88 custom Olympus cores, 1.2TB/s of memory bandwidth and up to 1.8x faster per-core performance on agentic workloads. In the NVL72 rack, Vera acts as host processor and links to pairs of Rubin GPUs through second-generation NVLink-C2C, sharing a memory architecture that Nvidia says keeps the accelerators fed.

The platform is arriving while memory is the scarcest part of the stack. Server DRAM contract prices have climbed sharply, with the fixed price for 64GB DDR5 server modules at $1,500 in mid-September, 5.5 times the level a year earlier, and Silicon Analysts reports that SK Hynix moved its 16-high HBM4 into volume shipping for Vera Rubin. Memory supply sets how quickly any of these systems can be built, which tempers what a per-rack throughput record means in the near term: the constraint on deploying Vera Rubin may prove to be HBM allocation rather than demand for its tokens.

AMD and Intel answer with scale and software

AMD's headline submission was not Vera Rubin's direct rival but a larger version of its current one. Crusoe, a first-time MLPerf submitter, ran 512 Instinct MI355X GPUs across 64 nodes on standard Ethernet, reaching 5.75 million offline tokens per second on GPT-OSS-120B and 2.90 million on DeepSeek-R1. AMD says those are the highest aggregate token throughputs submitted in MLPerf history and that the cluster scaled near-linearly from one to 64 nodes. The 512-GPU result is a different kind of claim from Nvidia's: it measures how much throughput an operator can assemble, not how much a single rack delivers.

AMD also leaned on software maturity. On the same eight MI355X GPUs it reported GPT-OSS-120B throughput up 28% offline and 38% in server within a single MLPerf cycle, and it says 72 MI355X GPUs in v6.1 beat the throughput that 94 of the same GPUs produced in v6.0. Intel's pitch was almost entirely software. It reported that gpt-oss-120B performance on the same four Arc Pro B70 GPUs improved 36% in server and 27% offline over v6.0, and that Xeon 6980P Llama 3.1-8B server throughput rose 2.4x on identical silicon. Intel remains the only standalone server CPU vendor in the suite, with five Xeon 6 SKUs and 35 CPU results.

What the preview label leaves open

Two caveats are worth holding alongside Nvidia's headline. The first is that these are vendor-run results on a fixed benchmark suite, even though MLCommons reviews them. MLPerf rewards throughput and latency on specific models under specific rules, and a system tuned for it is not automatically the best choice for a customer's own workload mix, cost target or latency budget. The second is the preview status itself: Vera Rubin NVL72 is not yet a production submission, and its real test comes in the round where it must be generally available with stated power, cooling and configuration requirements.

What to watch is straightforward. Vera Rubin's advantage should hold across a full round of production results, and the memory shortage is the variable most likely to decide how quickly any of this hardware reaches paying customers. AMD's and Intel's software gains show that the benchmark is not static, which means today's comparative ratios will look different in six months. For now, the clearest signal in v6.1 is not a single winner but a field where speed, scaling and software are all still moving at once.