NVIDIA open-sources the 550-billion-parameter Nemotron 3 Ultra, achieving a 5× increase in inference speed

On June 1, Jensen Huang announced two major updates on stage in Taipei: the next-generation GPU platform Vera Rubin has entered full production, and the NVIDIA Nemotron Alliance has released a 550-billion-parameter open-source mixture-of-experts model — Nemotron 3 Ultra. The former is hardware, the latter software, and the pairing is quite deliberate.

If last year’s Llama 3 and this year’s DeepSeek V3 raised the bar for open-source frontier models, NVIDIA’s latest move can be seen as stepping harder on the accelerator for open-source weights — and that accelerator sits squarely on its own Blackwell architecture.

A MoE Designed for “Long-Running Agents”

NVIDIA’s positioning for Nemotron 3 Ultra is straightforward: to provide top-tier intelligence for always-on agents, targeting three key application domains — code development, scientific research, and enterprise workflows.

With 550 billion parameters and a mixture-of-experts architecture, Nemotron 3 Ultra builds on the design philosophy of earlier Nemotron 3 models (the “Nano” version is already live on Hugging Face). Its sparse activation strategy follows the principle of “high total parameters, but keeping activated parameters per token in the tens-of-billions range.” This approach underpins its claim of up to 5× faster inference than comparable open-source models, while reducing usage costs by as much as 30%.

A bit of caution is warranted: NVIDIA hasn’t specified exactly which “comparable open-source frontier models” it means. Judging by parameter scale, it’s likely referring to MoE models such as DeepSeek V3 / R1 (around 671B parameters) or the larger models from the Llama 4 series. That “5×” figure probably comes from tests on Blackwell (B200 / GB200) hardware — the gap would shrink if run on H100 or older GPUs. Benchmark results for open-source models always depend on the hardware they run on.

This, however, highlights the cleverness of NVIDIA’s open-source strategy — the model itself is free, but its optimal inference performance is tightly coupled to NVIDIA’s hardware stack. With Vera Rubin in full production, Blackwell chips ramping up in scale, and continued updates to the Nemotron series, all three efforts align under one strategic plan.

Already Integrated with Mainstream Agent Frameworks

More noteworthy than simply releasing weights, Nemotron 3 Ultra includes extensive post-training engineering adaptation. Officially supported agent frameworks include:

• Hermes Agent
• LangChain Deep Agents
• OpenClaw
• OpenHands
• OpenCode

This means developers won’t need to manually align tool-call formats, planning-chain formats, or memory interfaces after acquiring the model — it can run directly inside mature frameworks such as LangChain Deep Agents or OpenHands. For enterprise deployment, this saves a substantial amount of engineering work.

Over the past year, the phrase “long-running agents” has shown up increasingly often in the messaging of companies like NVIDIA, Anthropic, and Cognition. Unlike single-turn QA or one-off code completions, long-running agents demand model consistency across task chains lasting tens of minutes to several hours — maintaining long contexts, stable tool calls, and robust error recovery are all difficult technical challenges.

Nemotron 3 Ultra’s MoE architecture is naturally well-suited to this: sparse activation means, under the same compute budget, it can support longer inference chains and reduce costs for continuous online operation.

CrowdStrike and Palantir Are Already Using It

NVIDIA presented two real-world adoption cases, both heavyweight examples.

CrowdStrike integrated the Nemotron model into its cybersecurity agent systems to perform continuous vulnerability scanning, risk assessment, and misconfiguration correction. This is a classic long-running scenario — security operations are 24×7 work traditionally done through rule engines plus human teams. An always-on agent that can run continuously, call tools, and write patches represents a tangible reduction in workload for security engineers.

Palantir’s implementation is even more interesting. It embedded Nemotron into its AI FDE (Forward Deployed Engineer) platform to create a closed-loop system within physically isolated enterprise networks. The data generated during agent interactions continuously feeds back for fine-tuning in specific business domains. This “data stays on-premise, model iterates locally” mode is one of the most significant advantages of open-weight models over closed API ones. If Palantir could only fine-tune via the GPT‑5 API, it could hardly support its hallmark high-security and high-customization use cases.

Not Just One “Ultra,” but a Whole Nemotron 3 Line

Zooming out, Nemotron 3 Ultra is simply the largest member of the family. The full lineup includes:

• Nemotron 3 Nano: lightweight; already live on Hugging Face; runs smoothly on edge or local devices
• Nemotron 3 Nano Omni: multi‑modal version; natively integrates vision and audio encoders; official claim of 9× performance improvement over the previous generation
• Nemotron 3 Super: released in March; previously billed as NVIDIA’s “most powerful open-weight model”
• Nemotron 3 Omni: multi‑modal integrated version combining audio, vision, and language
• Nemotron 3 Ultra: the star of this release; 550B‑parameter MoE
• Security and Speech‑Recognition Models: addressing two common enterprise agent gaps

From a product‑matrix perspective, NVIDIA isn’t just “shipping another big model.” It’s building a complete set of open‑source building blocks for enterprise‑grade agents. Nano covers edge applications; Super/Ultra handle cloud inference; auxiliary models bring safety and ASR; and integrations with LangChain, OpenHands, and similar high‑level frameworks complete the picture. The intended customer profile is clear: medium‑to‑large enterprises building their own AI infrastructure.

This differs slightly from Meta’s rationale for open‑sourcing Llama. Meta aimed to challenge OpenAI’s closed narrative; NVIDIA aims to bring every enterprise that wants to build AI onto its compute ecosystem. In this case, the open model acts as a “hook” — one that captures long‑term orders for Blackwell GPUs and NIM microservices.

Available for Download on June 4

According to NVIDIA’s timeline, Nemotron 3 Ultra will be distributed on June 4 via:

• Hugging Face
• ModelScope (Moda)
• OpenRouter
• build.nvidia.com (as NVIDIA NIM™ microservices)
• The inference platforms of major NVIDIA cloud partners

ModelScope is particularly convenient for developers in China, avoiding the need to connect directly to Hugging Face. For quick trials, the community will likely host API endpoints on OpenRouter as soon as it launches.

Points Worth Watching

First, how the “5× faster inference” claim pans out in practice. Support for MoE models varies widely across mainstream inference engines like vLLM and SGLang. Deploying a 550B‑parameter model isn’t trivial — you’ll need at least an 8×B200 setup or equivalent specs for smooth operation. The real performance numbers will come from the community.

Second, its potential impact on DeepSeek. Over the past six months, DeepSeek V3/R1 has dominated the open‑source MoE niche with strong cost-performance, Chinese-language capability, and inference efficiency. If Nemotron 3 Ultra can deliver clear differentiation in long tasks, tool usage, and agent scenarios, it would mark an intriguing head‑to‑head competition.

Third, the boundary of NVIDIA’s “open‑source” approach. Llama, Qwen, and DeepSeek are open‑source models from model-centric companies; NVIDIA is a hardware company open‑sourcing models — with different motivations and constraints. While model weights and parts of the training data are open (Nemotron Alliance has released several datasets), the optimization pipeline and engineering know‑how remain largely within NVIDIA’s NeMo framework. Whether this constitutes a healthy open ecosystem or simply another form of “open source as a sales funnel” will be told over time.

On the hardware side, Vera Rubin has shortened Blackwell-era assembly time from two hours to five minutes, doubling production capacity. On the software side, Nemotron 3 Ultra raises the ceiling for open-source MoE models once again. NVIDIA is transforming “selling GPUs” into “selling the full capability to build your own AI.”

That is the company’s biggest story today.

---

References

• NVIDIA releases 550‑billion‑parameter Nemotron 3 Ultra open‑source model, up to 5× faster inference than comparable frontier models – ITHome: official release details and partner collaborations
• Has NVIDIA become the new open‑source leader? Nemotron 3’s hybrid‑expert architecture boosts inference efficiency by 4× – Zhihu: detailed analysis of Nemotron 3’s architecture and activation strategy
• Nemotron model page on Hugging Face: NVIDIA’s official model repository; Ultra weights will be available here on June 4