We’re finally able to share all the assets and technical details of our Nemotron system, which achieved gold-medal-level performance at the IMO-26.
The release includes:
* Two specialized models that were part of the final ensemble system, alongside the generally available Nemotron3-Ultra checkpoint
* Two new large-scale datasets for SFT and RL to help models produce better natural-language mathematical proofs
* A set of 200 novel IMO-level mathematical problems
* A technical report with all the details needed to reproduce our work
We hope this release enables others to build on our work and helps advance open-source AI.
Everything is available at: huggingface.co/collections/n….
We're happy to announce that our Nemotron-based system scored 30 out of 42 points at the International Mathematical Olympiad (IMO) 2026, held last week. The IMO team officially graded the solutions and certified that the score was equivalent to a Gold Medal for a human participant.
1/5
Congratulations to the students who competed at the International Mathematical Olympiad (IMO) 2026. 👏
We put Nemotron 3 Ultra to the test to take on the same problems in the same time limit, with no internet or external tools.
The IMO team graded its solutions 30/42, above the 29-point gold threshold. 🥇
Our final submission consisted of an ensemble of three models, all based on the Nemotron-3-Ultra architecture. The models were specially trained to solve difficult mathematical problems and identify mistakes in proofs. The final system produced solutions within the same time limit as human participants, without access to the internet or external tools.
2/5
1
5
401
We released Nemotron 3 Ultra! It's quite strong across the board, but is especially good at math reasoning tasks compared to other open models. With enough TTC Ultra does really well on hard math competitions like IMO, USAMO and Putnam.
And as always, we aren't only releasing the model, but all datasets and recipes that make it this strong.
Nemotron-SFT-Math-v4: 550K solutions to "final answer" math problems with and without Python tool use huggingface.co/datasets/nvid…
Nemotron-Math-Proofs-v2: 80K solutions and verifications for hard mathematical proofs huggingface.co/datasets/nvid…
Nemotron-RL-Math-v2: 4K selected math problems with verified final answers. This dataset is much higher quality than any previous math RL set we released huggingface.co/datasets/nvid…
If you want to learn more details, check out our tech report research.nvidia.com/labs/nem…
NVIDIA Nemotron 3 Ultra is now live!
Frontier accuracy, 5X greater speed, 30% lower cost.
Deploy however you need - on-premise, on the cloud, or at the edge.
Model is live on HuggingFace under the OpenMDW 1.1 license.
youtube.com/watch?v=D8LIIvQV…
Nemotron 3 Super is out! It's really good and it will only get better from here.
And we release all the details - tech report, training code, training data, model weights. Everything you need to build a model like this yourself!
Announcing NVIDIA Nemotron 3 Super!
💚120B-12A Hybrid SSM Latent MoE, designed for Blackwell
💚36 on AAIndex v4
💚up to 2.2X faster than GPT-OSS-120B in FP4
💚Open data, open recipe, open weights
Models, Tech report, etc. here:
research.nvidia.com/labs/nem…
And yes, Ultra is coming!
Despite the small size, this is by far the best model we've released! And as always, we don't just release the model, but we release pretty much everything you need to reproduce it.
If you're interested in math reasoning, we have two new datasets for you to try.
- Nemotron-Math, which is a collection of 350K "final-answer" math problems and 7.5M natural language solutions generated by gpt-oss-120b (with and without Python tool use and with all 3 reasoning regimes). Training on this data you can easily get a model to 100% on AIME 24/25 if you use majority voting and Python TIR.
- Nemotron-Math-Proofs, which contains 580k natural language proof problems, 550k formalizations into theorem statements in Lean 4, and 900k reasoning trajectories from Goedel-Prover-v2 culminating in valid Lean 4 proofs (some theorems have multiple proofs). We weren't able to formalize all statements and we weren't able to prove all that we formalized, but we release everything, so that others can improve it. Doing simple SFT on this data can fully reproduce (and slightly improve) the accuracy of Goedel-Prover-v2 8B model on Lean benchmarks.
You can find the datasets as well as more details here:
- Nemotron-Math: huggingface.co/datasets/nvid…
- Nemotron-Math-Proofs: huggingface.co/datasets/nvid…
Today, @NVIDIA is launching the open Nemotron 3 model family, starting with Nano (30B-3A), which pushes the frontier of accuracy and inference efficiency with a novel hybrid SSM Mixture of Experts architecture. Super and Ultra are coming in the next few months.
What do we need to win IMO gold with open-weight LLMs next year? If we want to do it in natural language (like OpenAI and DeepMind did), we most likely need the following 3 components.
1. A scalable way to evaluate correctness of LLM-written proofs.
2. A massive test-time compute to generate, refine and verify proof candidates.
3. A large-scale RL run combining (1) and (2) to produce proofs for hardest training problems and reinforce correct reasoning.
We published a paper "Scaling Generative Verifiers for Natural Language Mathematical Proof Verification and Selection" exploring (1) and (2). Here are a few highlights.
2
2
65
How reliable are LLMs at evaluating mathematical proofs? To answer this question, we need a judge benchmark, i.e. a set of proofs with correct/incorrect labels. But does high accuracy on such a benchmark always mean an LLM judge has a good understanding of mathematical rigor? Not necessarily!
We take a recently released OpenProofCorpus dataset as a case study and show that a simple linear classifier on top of problem-proof embeddings gets 75.3% (higher than Claude-Sonnet-4). An even simpler baseline that predicts "incorrect" if a word "triangle" is in a problem (geometry is hard!) and "correct" if a string "---" is in a proof (o3 does that a lot) gets 65.1% (higher than GPT-4.1).
1
16
Now that AIMO-3 has been announced, I wanted to share a little ML puzzle that we had to deal with during AIMO-2. As with most things, it's somewhat obvious in retrospect, but took us a few hours to unravel during the competition. See if you can solve it!
One day, we trained a model and found that it always stops mid-word at exactly 16384 tokens, even if we asked for more. We knew the training and inference code was most likely correct, since we used it for many other models and we never had this problem before. So what was causing this? Here is all the context you need to piece it together.
- We were training Qwen2.5 models on the data generated by DeepSeek-R1. For this round only we also added new data generated by QwQ-32B that we never used before.
- When doing SDG we were asking for a maximum of 16384 tokens. You can see that it matches the stop position, so that's a clue, of course. But it did work fine before!
- After SDG, we would take all generations as strings and apply correctness filtering by extracting the value from the last \boxed{} and comparing to the GT answer. We discard anything that doesn't match or doesn't have \boxed{}.
- We then take all generations that passed correctness, wrap them in Qwen's chat template, and SFT on that.
When thinking through this, consider the following extra questions.
1. Why was this a problem only for this experiment and not before?
2. Why did the correctness filter not help here?
3. What is the fastest way to patch the issue?
4. What is a proper way to fix this?
4
132
Great blog post!
Fun fact: there is one case where this non-determinism at T=0 is actually a feature, not a bug. It helped us achieve slightly faster majority-voting generations with speculative decoding during AIMO-2.
We were generating several solutions with identical prompt and accidentally set T=0 in one of the runs. Surprisingly, we found that the majority voting accuracy with T=0 and T=0.6 was roughly the same, and the solutions were similarly diverse in both cases. However, using T=0 resulted in a higher acceptance rate for the draft model, making the generations slightly faster overall.
We added 4 new tutorials to NeMo-Skills documentation!
- "Inference with gpt-oss-120b using stateful Python code execution". Learn how to run gpt-oss models with a properly configured built-in Python tool including fully stateful code execution. Check it out to see how to get 100% on aime24/25 with majority voting!
- "Building an efficient inference engine for solving math problems" walks you through the inference pipeline we used in the AIMO-2 Kaggle competition. Learn how to speed up your inference 4x using fp8 and ReDrafter speculative decoding in TensorRT-LLM. See how to run LLM generations with async code execution and easily customizable early stopping.
- "Reproducing NVIDIA-Nemotron-Nano-9B-v2 evals" and "Reproducing Llama-Nemotron-Super-49B-V1.5 evals". See how to reproduce all reported evaluation numbers for those 2 great models released by NVIDIA. This is also a good place to learn about evaluation in NeMo-Skills in general!
Nemotron-Post-Training-Dataset-v1 is now open-sourced! The full dataset has 25M samples across multiple domains and can be used by anyone to improve their models!
The math, code and part of the stem subsets there is what we used to train OpenReasoning-Nemotron models. And the full 25M samples was used to create the new Llama-3.3-Nemotron-Super-49B-v1.5.
Get the data on HuggingFace: huggingface.co/datasets/nvid…
And see NeMo-Skills documentation for full details on how the math, code and science parts were created nvidia.github.io/NeMo-Skills…
Here is an interesting antipattern in the code generated by any of the frontier LLMs. Unless instructed otherwise, they try to avoid failing with error even when this is clearly the right behavior. Here is one example.
Prompt: Write a script to concatenate "input" and "output" keys in a jsonl to a new key "full_example" and resave to a new file.
All frontier LLMs give a variation of this solution. 🧵
Do you see what's the problem here? The code works just fine if there is no "input" or "output" key and it will simply use an empty string instead. Consider what happens if you accidentally run it on a malformed file that doesn't have input / output fields for some entries (e.g. you made a typo in the key name, or some other bug caused this). You will get a valid output file, and the script will finish without any error. You might then use this new file to e.g. train a model, get bad results (because part of the data is corrupted), conclude that your experiment doesn't work and move on.
Such silent problems can be really hard to catch! It's much better if the code just fails with an explicit error rather than misbehaving silently.
1
1
162
Important to keep this in mind when AI systems participate in any human competitions. Still an incredible achievement by OpenAI of course
1
4
497
Wanted to share a few more interesting observations about this release.
First, a quick clarification, this is very much a **research** model release. We only trained models on math, code and science reasoning tasks with no instruction tuning or RLHF. While they should be good at solving reasoning tasks, they might not be able to hold multi-turn conversation or be useful general assistants without further training. The main goal for building these models was to validate the value of the new data we produced and see how far we can push results with just SFT.
Now here are 2 things that we found interesting when developing these models.
1. If you look at the results of 1.5B, it's actually not that great. E.g. OpenMath-Nemotron-1.5B (our previous math-only model release) gets 49.5 on AIME25, while this new model gets 45.6. But if you look at 7B (or bigger) models, there is a huge improvement. OpenMath-7B model was getting 61.2, but OpenReasoning-7B gets 78.2! So 1.5B got a little worse, but 7B got almost 20% better after training on exactly the same data. While we don't know the reason for sure, we suspect that it's because 1.5B model isn't very consistent with handling long context generations. Our previous data was only 16K output tokens, but this round we doubled it to 32K and it seems that it might be too much for 1.5B and it's not able to keep the reasoning consistent. Would be nice to do a more in-depth analysis here.
2. In our previous OpenMath release we also used some TIR data to teach models how to use Python. We didn't have time to regenerate this data with new R1, but we decided to include a little bit of the old TIR data into the current blend to see what happens. Our hope was that maybe during training the model would still learn how to use Python but keep the much better reasoning from the new CoT samples. Yet this didn't happen - if you evaluate OpenReasoning models with TIR mode, you will see that they are basically the same as OpenMath models, which is much worse than the new models with CoT. One way to look at this is that the models basically learned two sets of behaviors - one with tools, but worse reasoning and one without tools, but great reasoning, and there is no interpolation between the two. Would be interesting to see if this can be fixed with some on-policy RL in TIR mode!
We've released a series of OpenReasoning-Nemotron models (1.5B, 7B, 14B and 32B) that set new SOTA on a wide range of reasoning benchmarks across open-weight models of corresponding size.
The models are based on Qwen2.5 architecture and are trained with SFT on the data generated with DeepSeek-R1-0528.
A few highlights 🧵
We've released a series of OpenReasoning-Nemotron models (1.5B, 7B, 14B and 32B) that set new SOTA on a wide range of reasoning benchmarks across open-weight models of corresponding size.
The models are based on Qwen2.5 architecture and are trained with SFT on the data generated with DeepSeek-R1-0528.
A few highlights 🧵
Here is a video on how to get started with NeMo-Skills. We show a simple pipeline that improves a weak math reasoning LLM by up to 20% on AIME benchmarks. Everything is built fully end-to-end—SDG, model training, benchmark evaluation. All commands take ~2 hours on 8 A100 GPUs.
NeMo-Skills makes it easy to build powerful training and inference pipelines—seamlessly switching between frameworks and scaling from local prototyping to massive Slurm-based experiments with just a single line change.
In this video, we’ll walk through how you can get started with Nemo-Skills to:
✅ Run LLM inference, convert checkpoints, train and evaluate models
✅ Iterate quickly from data generation to model training and evaluation
✅ Connect multiple pipeline stages into one unified script
Whether you're scaling up or just getting started, NeMo-Skills helps you move fast, experiment freely, and build smarter.
🎥 nvda.ws/45PtyZV
Text version of the same tutorial is here developer.nvidia.com/blog/ho…
And more details are in the repo github.com/NVIDIA/NeMo-Skill…
1
65
NeMo-Skills is the library behind many of the recent LLM releases from Nvidia - OpenMathReasoning, OpenCodeReasoning, LlamaNemotron.
NeMo-Skills makes it easy to build powerful training and inference pipelines—seamlessly switching between frameworks and scaling from local prototyping to massive Slurm-based experiments with just a single line change.
Get started with our technical blog: nvda.ws/4lsrUBT
1
2
106