NVIDIA Vera Rubin Launched: A New Generation of Computing Foundation for Agentic AI

NVIDIA has brought the Vera Rubin platform into the spotlight—not a PowerPoint, but real mass production and delivery.

According to the official update on July 16, seven Vera Rubin chips are now in full production, five rack configurations and a complete AI factory reference design are ready, and shipments to top-tier clients such as AWS, Anthropic, Meta, Microsoft, OpenAI, xAI, CoreWeave, Oracle, Alibaba, and ByteDance are set to begin in the second half of the year. The core message this time can be summed up in one sentence: maximize the amount of intelligence each dollar buys at the post-training and agentic reasoning stages.

The term “intelligence per dollar” is a new metric NVIDIA introduced itself. In plain language, it acknowledges that the GPU arms race of model training is now in its second half—the real cost driver for clients is inference and post-training loops once agents are running 24/7. Rubin is built for this battlefield.

From “Next-Gen GPU” to “Next-Gen Datacenter”

Let’s clarify an easily overlooked point: Vera Rubin is not the name of a new GPU, but the name of an entire supercomputer.

The seven chips that make up this system are:

• Vera CPU: 88-core, proprietary Olympus architecture, a server CPU—NVIDIA has finally incorporated this into its own product line.
• Rubin GPU: built on TSMC’s N3P process, delivering 50 PFlops inference / 35 PFlops training at NVFP4 precision per card.
• NVLink 6 Switch: sixth-generation NVLink, enabling high-speed interconnects between chips.
• ConnectX-9 SuperNIC: next-generation super network card.
• BlueField-4 DPU: responsible for storage and KV cache offloading.
• Spectrum-6 Ethernet Switch: built-in CPO silicon photonics technology.
• Groq 3 LPU: the Easter egg—NVIDIA’s first integrated product after acquiring Groq for $20 billion late last year.

These seven chips are assembled into five rack types: Vera Rubin NVL72 GPU rack, Vera CPU rack, Groq 3 LPX inference rack, BlueField-4 STX storage rack, and Spectrum-6 SPX network rack. Alongside the Omniverse DSX Blueprint digital-twin design, it forms a ready-to-deploy AI factory.

Jensen Huang described it as a “generational leap.” Stripping away the marketing tone, the structural change is indeed real: Vera CPU and Rubin GPU are co-packaged with a 1.8 TB/s NVLink-C2C interconnect—effectively eliminating PCIe. This physically removes the long-standing bottleneck between CPU and GPU—an essential breakthrough for agents that involve high-frequency “model inference + tool invocation + state feedback” cycles.

Vera Rubin NVL72: Impressive Specs—But Usage Matters

The product most comparable to the Blackwell GB200 NVL72 is the Vera Rubin NVL72: 72 Rubin GPUs + 36 Vera CPUs, fully interconnected via NVLink 6, with ConnectX-9 and BlueField-4 included.

NVIDIA’s own comparison figures show:

1. For training models of the same MoE scale, requires only 1/4 the GPUs of Blackwell.
2. Up to 10× higher inference throughput per watt.
3. 1/10 inference cost per token.

That third number is key. Why have vendors stopped stressing “X× faster training” and started talking about token cost? Because clients no longer buy that story. The costs that keep CFOs awake are the daily inference bills from products like Claude Code, Cursor, and Perplexity. A single agent workflow may trigger dozens of model calls, consuming millions of tokens; whatever was saved on one training run gets eaten up by inference within weeks. For the first time, Rubin calculates “post-training + inference” as one combined cost structure—a sensible move.

Total internal rack bandwidth hits 260 TB/s—a number that sounds wild because it’s several times the total cross-border bandwidth of the global Internet. The cost of data movement within a rack is now low enough to be “free-flowing”—this is the physical foundation enabling agents with context windows of hundreds of thousands of tokens to respond in real time.

Groq 3 LPX: NVIDIA Finally Builds an “Anti-GPU” Inference Chip

This was perhaps the most intriguing part of the launch.

Late in 2025, NVIDIA spent $20 billion acquiring Groq’s LPU architecture under strategic licensing and deep integration—widely seen as a “defensive acquisition” at the time. Now, with the Rubin platform in place, the intent is clear—NVIDIA wants to decouple and sell the training and inference markets separately.

The Groq 3 LPX is a standalone inference accelerator rack featuring:

• 256 LPU processors per rack
• 128 GB on-chip SRAM
• 640 TB/s aggregate bandwidth
• Fully liquid-cooled, built on the MGX infrastructure

Technically, LPU abandons speculative GPU designs like branch prediction, instruction reordering, and complex cache management. Instead, it uses a deterministic pipelined architecture, offloading all hardware complexity to the compiler layer. The direct benefit: constant, jitter-free execution time.

That’s essential for autonomous driving, high-frequency trading, and long inference chains in agents. One agent chain might involve dozens or hundreds of model calls—any latency jitter in one call can collapse the entire sequence. With LPUs keeping all weights in on-chip SRAM and GPUs using HBM for trillion-parameter models, both are fused into the CUDA ecosystem via NVFusion—creating a clean separation where “GPUs handle training, LPUs handle inference.”

Interestingly, NVIDIA claims that when LPX and Vera Rubin NVL72 are deployed together, Rubin GPUs and LPUs co-process every model layer to jointly accelerate decoding speed. That’s not simple train/infer separation—it brings the LPU into the main inference path as a co-processor. 35× improvement in inference throughput per megawatt—if proven in real tasks, that could redefine pricing throughout the inference market.

Vera CPU Rack: The Underestimated Component

While GPU and LPU news took the spotlight, the Vera CPU rack marks NVIDIA’s formal entry into the server CPU market.

Key specs:

• 256 Vera CPUs per rack
• Supports 22,500+ concurrent sandboxed environments
• Liquid cooling, MGX modular design
• Compared with traditional CPUs: 2× efficiency, 50% faster

Those “22,500 concurrent sandboxes” are tailor-made for the agent era. Think of products like Claude Code or Cursor—each code execution, tool call, or evaluation needs an isolated sandboxed environment. In the past, Xeon or EPYC handled that; now NVIDIA is taking over directly, offloading orchestration, evaluation, and data-handling tasks related to agents to CPU-level hardware.

Client partners include Alibaba, ByteDance, Meta, Oracle, CoreWeave, Lambda, Nebius, and Nscale—spanning from hyperscale clouds to new GPU-cloud startups. Intel and AMD’s presence among large-scale AI customers has effectively been pushed to the margins.

Post-Training: The Real Battlefield

The Rubin platform’s flagship metric is “post-training intelligence per dollar,” a phrase worth unpacking.

From 2023 to 2025, the industry’s compute anxiety centered on pretraining—how many GPUs are needed to train a GPT-4-scale model. By 2026, the narrative shifted—pretraining players have consolidated into fewer than ten, while everyone else’s task is to perform post-training on open-source baselines. RLHF, DPO, GRPO, rejection sampling, synthetic data, reward model training—all of these workloads share one trait: they are inference-intensive. Each training step involves extensive inference to generate candidates before applying gradient updates.

That’s why NVIDIA emphasizes “post-training” instead of “training”—because post-training is the most inference-like part of training itself. Rubin’s design—large HBM capacity, massive interconnect bandwidth, BlueField-4 KV cache offload, and the Inference Context Memory storage layer—all target this workload precisely.

Transformer Engine 3.0 also received an update, optimized for NVFP4 precision and long context handling. For developers, this means each “generate → score → update” iteration in post-training loops can theoretically run much faster and cheaper than on Blackwell, given the same token counts.

A Few Sober Realities

Let’s balance the hype with some caution:

First, mass production doesn’t mean availability. Rubin only begins shipments in the second half of this year, and ramp-up won’t stabilize until early 2027. Early batches will likely go to hyperscalers and top AI labs; smaller cloud providers will wait longer. Blackwell’s 2025 was essentially “sold out with cash on the table”—Rubin will likely repeat that.

Second, “10× cost reduction” is the official figure. Marketing numbers are always optimistic. Blackwell’s advertised 25× efficiency uplift translated to roughly 3–5× in practice. Rubin’s 10× reduction may mean 3–4× real-world savings—still substantial, but don’t build financial models on the official number.

Third, the Groq LPU path isn’t yet proven at scale. Deterministic architecture is unbeatable for small, static contexts, but once models change frequently and contexts become complex, SRAM capacity will quickly become a bottleneck. NVIDIA positioning it as a “co-processor” rather than a “replacement” already signals this limitation.

Fourth, the software ecosystem is both moat and burden. CUDA keeps customers locked in but forces Rubin to maintain backward compatibility with all Blackwell-era inference frameworks, quantization schemes, and compiler optimizations. Fast hardware iteration always tests software’s ability to keep pace.

What This Means for Developers

If you’re training models, no reason to worry for now—keep using your current H100/H200/B200 setup; Rubin’s price won’t be low this year.

If you build agent or AI application products, the key to watch is when inference API pricing begins to relax. Historically, it takes about 9–12 months for each new NVIDIA architecture to pass manufacturing cost reductions through to major cloud providers. If Rubin ships as scheduled, by around Q2 2027, mainstream inference providers will likely lower per-token prices—a window of opportunity for agent developers to restructure cost models.

If you work on post-training, distillation, or fine-tuning, pay attention to the Vera CPU + BlueField-4 STX combo—it offloads KV cache management, sandbox isolation, and tool orchestration to hardware. The long complaint about high engineering complexity in RLHF loops could finally yield to a cleaner abstraction.

Huang predicts combined Blackwell + Rubin revenue will reach $1 trillion by the end of 2027—double the $500 billion forecast given last October. Whether the capital markets believe it or not, NVIDIA’s technology roadmap has clearly shifted: it is no longer just a “GPU maker.” It now sells entire AI factories, with chips merely forming the foundation.

For the industry, the generation’s significance isn’t merely higher speed—it’s that NVIDIA has officially defined the “intelligent token” as an accountable economic unit. Whoever can lower its cost and raise its output will hold pricing power in the Agentic AI era. Rubin is NVIDIA’s first move to seize that power.

References

• Jensen Huang drops a bomb at CES: Next-gen Rubin architecture cuts inference costs by 10× (Zhihu) — Compilation of Rubin platform details on inference costs, Transformer Engine, and NVLink interconnects.