NVIDIA Nemotron 3 Embed Takes the Top Spot on the RTEB Leaderboard

NVIDIA Has Conquered Retrieval, Too

On July 15, NVIDIA released evaluation results for its Nemotron 3 Embed series on Hugging Face — the 8B model ranked first on the RTEB (Retrieval Embedding Benchmark) leaderboard. This position has long been dominated by OpenAI’s text-embedding-3-large, Cohere’s embed-v3, and various Voyage AI models. While it's not new for NVIDIA to make embedding models, this is the first time an open‑source and a closed‑source leaderboard have both been topped at once.

If you’ve been working on RAG or Agent retrieval recently, this is worth two minutes of your time.

Why RTEB and Not MTEB

Let’s clarify what benchmark this is about. For several years, embedding‑model developers have been focused on MTEB (Massive Text Embedding Benchmark) — wide task coverage, multilingual, and highly recognized by the community. But after years of use, the flaws became clear: much of its data leaked into training sets, meaning many models’ high scores came from overfitting and didn’t hold up in production.

RTEB, launched by Hugging Face this year, was rebuilt specifically to solve MTEB’s pain points:

• Most datasets remain private – model authors can’t access test sets, so no targeted optimization
• Focuses solely on retrieval, excluding classification and clustering tasks that are irrelevant to RAG
• Covers 20 languages and 11 professional domains – law, finance, healthcare, code, and other business‑critical fields

In short, RTEB’s goal is to make the leaderboard honest again. Ranking first under this standard is far more meaningful than scoring well on MTEB. Nemotron 3 Embed 8B ranks first overall, as well as in legal and financial domains, and top three in code retrieval. This isn’t a “+0.3‑point in some narrow task” type of win — it performs strongly across the board.

What Exactly Is Nemotron 3 Embed

NVIDIA released two sizes this time:

• Nemotron 3 Embed 8B – an embedding model fine‑tuned from the Qwen3 series, producing 4,096‑dimensional vectors; the star of this RTEB run.
• Nemotron 3 Embed 1B – smaller and faster, available in BF16 and NVFP4 precision versions, the latter optimized for vLLM inference.

Several design choices stand out:

Bidirectional Attention + Mean Pooling.
The 1B model uses a transformer with bidirectional attention masks and mean pooling over token representations. This is a return to mainstream embedding practice: many had experimented with causal models for embeddings over the past two years with unstable results. Nemotron goes back to the reliable encoder‑style approach.

NVFP4 Quantization.
The 1B NVFP4 version is post‑training quantized using NVIDIA’s Model Optimizer (modelopt v0.45), purpose‑built for vLLM 0.25.0. On H100/H200/B200 GPUs, this combination further lowers embedding inference costs. Previously, 8B embeddings often hit throughput bottlenecks for bulk vectorization. Now, the 1B NVFP4 model should meet most use cases, while allowing multiple replicas per GPU.

OpenMDW‑1.1 License.
This isn’t Apache 2.0 — it’s NVIDIA’s own open‑model license. Commercial use is allowed, but there are clauses restricting redistribution and model output use. Enterprises should have legal review this before deployment to avoid compliance issues.

Why Retrieval Models Are in the Spotlight Again

From 2024 through the first half of 2025, industry attention has fixated on generative models — context window length, reasoning, agent orchestration. Retrieval was assumed to be a “solved” problem: just plug in OpenAI’s embedding‑3 or a BGE variant and call it a day.

But this year, every team building Agents learned the same painful lesson:
retrieval quality determines the agent’s ceiling.

A realistic example: your Agent has 200 tools and 50,000 internal documents. A user asks, “The three SKUs with abnormal return rates in the East China region last quarter.” Even the best generative model won’t help if retrieval matches “return” to “refund” and drops “East China.” A model that understands semantic nuance, handles multilingual input, and professional terminology well delivers much more value than simply spending 5× more tokens on brute‑forcing reasoning with an LLM.

Nemotron 3 Embed is built exactly for this. NVIDIA describes it as part of the “Agentic AI workhorse” roadmap in the Nemotron 3 series — alongside Nemotron 3 Super (120B MoE, 12B active) for long‑context and tool‑use scenarios. Retrieval, generation, and tool invocation — NVIDIA’s message is clear: they’re building the whole Agent stack.

How It Compares to Others

Let’s compare Nemotron 3 Embed 8B against mainstream options:

| Model | Dimension | Open Source | RTEB Overall | Positioning | |--------|------------|-------------|--------------|--------------| | Nemotron 3 Embed 8B | 4096 | ✅ (OpenMDW) | #1 | Agent retrieval, multilingual, professional domains | | OpenAI text-embedding-3-large | 3072 | ❌ | Upper-mid | General-purpose, easy to use | | Cohere embed-v3 | 1024 | ❌ | Upper-mid | Multilingual, enterprise | | Voyage-3 | 1024 | ❌ | High | General retrieval | | BGE-M3 | 1024 | ✅ | Mid | Chinese, multitask | | llama-embed-nemotron-8B | 4096 | ✅ | High (Multilingual MTEB #1) | Cross-lingual |

Key takeaways:

4096 dimensions aren’t all good.
Higher dimensions mean better retrieval quality but also higher storage and latency costs. For 100 million vectors, 1024 D vs. 4096 D means 4× storage and ~3–4× longer ANN indexing time. So the 1B model exists for good reason — most business cases are fine with it; the 8B is reserved for extreme‑quality cases.

More open‑source choices.
Previously, Chinese teams basically defaulted to BGE‑M3. Now with Nemotron 3 Embed 8B and llama‑embed‑nemotron‑8B (which led the multilingual MTEB in October), the open‑source ceiling just got higher. Expect pressure on BGE’s team.

Closed‑source API moats are shrinking.
OpenAI’s embedding API works well but is expensive ($0.13 per million tokens, even in batch) and can’t be self‑hosted. Security‑sensitive enterprises will be recalculating — especially now that open models can actually take the top spot.

Deployment Reality

The 1B NVFP4 version can be used with vLLM like this:

from vllm import LLM

MODEL = "nvidia/Nemotron-3-Embed-1B-NVFP4"
llm = LLM(model=MODEL, task="embed")

texts = [
    "上季度华东区退货率异常的三个 SKU",
    "Q2 East China region SKU return rate anomaly",
]
outputs = llm.embed(texts)
vectors = [o.outputs.embedding for o in outputs]

If you’re using Transformers or Sentence Transformers, NVIDIA recommends the BF16 version (nvidia/Nemotron-3-Embed-1B-BF16); the NVFP4 checkpoint is for vLLM only. The split is clear: BF16 for training / fine‑tuning, NVFP4 for production inference.

The 8B model’s inference cost deserves careful planning. With 4096 dimensions, 8B parameters, and bidirectional attention, a single H100 GPU (batch 128) should reach roughly 3,000–4,000 tokens/s (unofficial estimate based on comparable benchmarks). It’s feasible for daily offline vectorization but for online expansion queries, the 1B variant is a better fit.

The Less‑Perfect Parts

No surprise in long context.
No public info yet shows Nemotron 3 Embed handling context longer than 8K tokens. Long‑document retrieval still needs chunking; it’s not one of those 32K or 128K‑context models.

Insufficient standalone Chinese evaluation.
RTEB is multilingual; Chinese results are strong but not separately broken out. For production in Chinese enterprise settings, run your own A/B test on business data instead of trusting the leaderboard alone.

8B is overkill.
For most “good‑enough” retrieval tasks, 8B is excessive. Most teams will pick the 1B version; 8B is for cases where retrieval quality trumps everything — e‑discovery, medical literature search, patent retrieval, etc.

What This Means for Agent Developers

Zooming out:

By mid‑2025, the Agent community largely agrees that generation is good enough, but bottlenecks remain in orchestration and retrieval. Frameworks like LangGraph, Agno, and CrewAI have matured on the orchestration side; retrieval, meanwhile, has lacked a truly SOTA open‑source option — until now. Nemotron 3 Embed fills that gap.

For RAG product teams, try this path: replace your current BGE or OpenAI embedding with the 1B version, run an A/B test, and check Recall@10 on real data. If it improves by ≥5%, consider upgrading to 8B.

For vertical Agent teams (law, finance, etc.), the 8B version may deliver even greater gains, since RTEB domain scores show Nemotron leading by wider margins there.

For OpenAI Hub users, if your Agents use GPT, Claude, or Gemini for generation, a single key via API is enough. For embeddings, self‑host Nemotron 3 Embed with vLLM, or wait for a managed service — decoupled retrieval‑generation deployment is already the mainstream Agent architecture.

References

• nvidia/Nemotron-3-Embed-1B-NVFP4 Model Card — official page for the 1B NVFP4 quantized model, includes vLLM deployment and license info
• NVIDIA Nemotron 3 Embed Ranks #1 Overall on RTEB — NVIDIA’s official blog post with full benchmark data
• Llama-Embed-Nemotron-8B Multilingual MTEB Leaderboard Overview — another embedding model NVIDIA released in October for comparison
• NVIDIA Nemotron 3 Series: Efficient Models for Multi-Agent Systems — Zhihu article analyzing the overall Nemotron 3 lineup (Nano / Super / Embed)