NVIDIA AI Releases Nemotron 3 Embedded: 8B Benchmark Open Embedded Cluster Ranked #1 in RTEB

Embedding models determine which episodes the agent has seen. NVIDIA released Nemotron Model 3 Embed working on that layer. It targets RAG for production scale, agent recovery, code recovery, and agent memory.

What is Nemotron 3 Embed?

The model cluster includes three open test areas. Nemotron-3-Embed-8B-BF16 accuracy-first option. Nemotron-3-Embed-1B-BF16 carries the same design into a smaller footprint. Nemotron-3-Embed-1B-NVFP4 Blackwell-optimized 4-bit method.

All three are transformer codes that they were trained on to hide dual attention. The last embedding appears integration ratio token level representation. The maximum length of the sequence is 32,768 tokens for the whole test area.

Each model was tested in 34 languages. All three are in charge OpenMDW License Agreement, version 1.1 (OpenMDW-1.1). Notably, the bases are Mistral models. 8B is built with Ministral-3-8B-Instruct-2512. Both variants of 1B use Ministral-3-3B-Instruct-2512.

Working

Nemotron-3-Embed-8B-BF16 standards #1 overall on RTEB (as of July 17 2026), Restorative Embedding Benchmark. The assessment covers its 16 public works. Each figure below is an average of NDCG@10, for models of 4096 sequence lengths.

Two gaps should be noted. 1B earns 10.4 more RTEB points llama-nemotron-embed-vl-1b-v2the foundation of the previous generation. Separately, NVFP4 scores 0.38 RTEB against its parent BF16, or 99.5% retention.

How to build a 1B model?

Those 1B points are from the squeeze pipe, not the little training. So was the parent nemotron-3-embed-3bpruned and dissected in two repeated rounds.

First, the 3B parent was pruned to 2B using NVIDIA ModelOpt mcore_minitron Neural Architecture Search (NAS). Searches include hidden width, FFN size, attention heads, and depth. It then selects the candidate with the best Pareto top 10. A 50k domain rating chorus found those candidates.

Next, the 2B model was refined from the well-configured 8B embedding model. The distillation is combined cosine distance loss (COS) again mean square error (MSE) loss. Data collection was multi-lingual and multi-domain. Finally, the same process was repeated to produce the 1.14B test area.

NVFP4 Serving Tradeoff

The compression then proceeds to the feed format. Quantization weights and activation of only linear layers, targeting NVFP4 data type. A research team was used nvidia-modelopt v0.45.0. Quantization-Aware Distillation (QAD) followed, especially to restore the accuracy of long input.

The measurement used 512 samples: 256 questions and 256 episodes abisee/cnn_dailymail. QAD training used 20k samples.

The research team reports NVFP4 in Blackwell’s delivery up to 2x more than BF16 output. It ends 99%+ BF16 retrieval accuracy. The NVFP4 card also features dynamic embedding sizes. You can crop a 2048-d vector from scratch to 1024 or 512 dimensions. Reset to normal after that.

Interactive Descriptor: A Five-Stage Retrieval Method

Before touching the code, look at the running method. It updates start-up, double-coding, central integration, L2 normalization, and dot product scores. Scores are derived from the expected production of each card.

Deployment Matrix

As that walkthrough shows, checkpoints don’t share runtime methods.

Alongside the benchmarks, NVIDIA’s research team released an optimized NIM microservice in the 1B model. The Rust-based NIM matches or exceeds the vLLM benchmark on the GB200 and RTX PRO 6000. NVIDIA tested input sequence lengths of 256 and 1024. Separately, NVIDIA NeMo AutoModel recipes include fine tuning and distillation.

Using the Code

With those methods in mind, first things first. Questions are taking query: and the documents take passage: . The embedding is L2-normal, so the dot product is equal to the cosine similarity.

# pip install --upgrade "transformers>=5.2.0" "sentence-transformers>=5.4.1"
import torch
from sentence_transformers import SentenceTransformer

QUERIES = ["How can someone reduce exposure to pollen during allergy season?"]
DOCUMENTS = ["People with pollen allergy can reduce exposure by staying indoors "
             "on dry, windy days, avoiding early-morning outdoor activity, and "
             "going outside after rain when pollen levels are lower."]

model = SentenceTransformer(
    "nvidia/Nemotron-3-Embed-8B-BF16",
    device="cuda",
    model_kwargs={"dtype": torch.bfloat16,
                  # use "sdpa" if FlashAttention-2 is unavailable
                  "attn_implementation": "flash_attention_2"},
    processor_kwargs={"padding_side": "left"},
)
model.max_seq_length = 32768

q = model.encode_query(QUERIES, batch_size=1, convert_to_tensor=True)
d = model.encode_document(DOCUMENTS, batch_size=1, convert_to_tensor=True)
print(model.similarity(q, d))  # card's published q[3]/d[3] score: 0.8008

encode_query again encode_document read the saved commands. So you never add prefixes manually. For worship, /v2/embed uses them from input_type in turn:

vllm serve nvidia/Nemotron-3-Embed-1B-NVFP4 
  --max-model-len 4096 
  --max-num-batched-tokens 4096 
  --max-cudagraph-capture-size 4096

import numpy as np, requests

def embed(input_type: str, texts: list[str]) -> np.ndarray:
    r = requests.post(
        "
        json={"model": "nvidia/Nemotron-3-Embed-1B-NVFP4",
              "input_type": input_type,          # "query" or "document"
              "texts": texts,
              "embedding_types": ["float"],
              "truncate": "END"},
        timeout=120,
    )
    r.raise_for_status()
    return np.array(r.json()["embeddings"]["float"], dtype=np.float32)

scores = embed("query", QUERIES) @ embed("document", DOCUMENTS).T

Use Cases with examples

• Business search in multiple languages: The support team identifies Hindi, Japanese, and English tickets together. Because the retrieval is multilingual, a German query can produce a Japanese decision note.
• Retrieving the code: Training is included coir_apps, coir_cosqa, synthetic_text2sqland the SWE bench. So the natural language lookup to the code is close to distributed.
• Agent memory: A limit of 32,768 tokens allows an agent to embed snippets of a long conversation without aggressive cramming.
• A high quality RAG: Serve 1B-NVFP4 with high volume recall, and move difficult questions to 8B. Because the range varies, this requires two pointers.

Key Takeaways

• Nemotron-3-Embed-8B-BF16 hits #1 on RTEB at 78.46 avg NDCG@10.
• The three open test areas are 8B BF16, 1B BF16, and 1B NVFP4.
• NVFP4 maintains 99%+ accuracy of BF16 up to 2x Blackwell output.
• 1B came from ModelOpt NAS distillation and COS+MSE distillation from 8B.
• All checkpoints use OpenMDW-1.1 and support 32,768 token input.

Check it out NVIDIA presents a post on Hugging Face, Nemotron Collection 3 Embed, 8B-BF16 card, 1B-BF16 card again 1B-NVFP4 card. Also, feel free to follow us Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to Our newspaper. Wait! are you on telegram? now you can join us on telegram too.

Need to work with us on developing your GitHub Repo OR Hug Face Page OR Product Release OR Webinar etc.? contact us