Guilin Liu (@GuilinL) profile picture

Principal Research Scientist & Team Leader at NVIDIA.

Santa Clara, CA
Joined September 2016
Photo by Guilin Liu (@GuilinL)Photo by Guilin Liu (@GuilinL)Photo by Guilin Liu (@GuilinL)Photo by Guilin Liu (@GuilinL)Photo by Guilin Liu (@GuilinL)Photo by Guilin Liu (@GuilinL)
I joined NVIDIA in 2017 after earning my Ph.D. and never left. At GTC 2018 Taiwan, I had the privilege of demonstrating image inpainting technology alongside Jensen during his keynote. I was less than a year into my career at the time, and it was his patience, encouragement and understanding that helped me overcome my nerves and deliver the demo to an audience of thousands. NVIDIA is where it is today because of Jensen’s leadership, vision, wisdom and resilience. It has been a heartbreaking day.
65
102
7
1,742
70,190
Nvidia’s Nemotron 3.5 Super VL is now the leading US open weight model, scoring 26 on the Artificial Analysis Intelligence Index Nemotron 3.5 Super VL achieves twice the intelligence of its predecessor, Nemotron 3 Super (13), and does so using ~60% fewer output tokens per task It launches alongside Nemotron 3.5 Bolt 74B-A9B 24GB, the first model in its new Bolt family, which scores 21 on the Intelligence Index Congratulations to @NVIDIAAI on the releases! Key Takeaways: ➤ Leading US open weight model: Nemotron 3.5 Super VL scores 26.5, edging out Thinking Machines' Inkling Small (25.7) and ahead of NVIDIA's Nemotron 3 Ultra (23) and Meta's Muse Glimmer (17) ➤ Bolt comes within 2 points of Ultra: Nemotron 3.5 Bolt 74B-A9B 24GB scores 21, 1.5 points behind Nemotron 3 Ultra 550B A55B (23). The 2 bit memory has about a seventh of Ultra's parameters (74B vs 550B) and a sixth of its active parameters (9B vs 55B) ➤ Super 3.5 VL near lossless in NVFP4: Nemotron 3.5 Super VL releases in NVFP4, scoring within 0.1 points of its BF16 weights on the Intelligence Index, with no component moving by more than 2.6 points Nemotron 3.5 Super VL details Context window: 1M tokens Modalities: Vision and text input; text output Parameters: 120B / 12B Nemotron 3.5 Bolt 74B-A9B 24GB details Context window: 256k tokens Modalities: Vision and text input; text output Parameters: 74B / 9B
90
85
51
1,013
170,370
Guilin Liu retweeted
Meet NVIDIA Nemotron 3.5 Super VL Our new open model combines reasoning and tool use with text, image, document, and video understanding for long-running AI agents. 120B total parameters with 12B active 🧵
🤖 Made with AI
291
549
365
4,444
614,032
LocateAnything is the #1 trending model on HF. Great work from our interns and collaborators. research.nvidia.com/labs/lpr…
So much great work lately from Nvidia, the "King of American Open-source AI"! - Crossed 1,000 total public repositories on @huggingface (820 models, 249 datasets & 57 spaces) & almost 60,000 followers - Current #1 trending model on HF with LocateAnything and #5 trending with PiD - Announced that they're adopting the @linuxfoundation OpenMDW framework - Released Cosmos 3, Omnimodal World Models for Physical AI & Alphamayo 2 Super, an open model for autonomous driving - Announced the release soon of Nemotron 3 & work on Nemotron 4 Thank you @nvidia for all the work you're doing for the ecosystem and open-source AI. Can't wait for the next few months!
4
1,659
Guilin Liu retweeted
Nemotron 3 Ultra is now the best open weight model on pinchbench.com/ 💚
16
39
4
378
57,102
We presented Parallel Box Decoding which improves both decoding efficiency and localization accuracy for vision-language grounding. Please check out more examples and demo through the project page: research.nvidia.com/labs/lpr…
This #CVPR2026 paper from our research team is trending #1 on @HuggingFace 🤗 Meet LocateAnything: a vision-language detection model that rethinks bounding box prediction. For AI agents and robots, “seeing” is only useful if a model can pinpoint where something is fast enough to act. Trained on 138M high-quality samples, LocateAnything decodes bounding boxes in parallel instead of one coordinate at a time, improving localization accuracy while dramatically increasing throughput for visual grounding and detection. Project page: nvda.ws/4dKSohb
913
Guilin Liu retweeted
Meet Nemotron 3 Nano Omni 👋 Our latest addition to the Nemotron family is the highest efficiency, open multimodal model with leading accuracy. 30B parameters. 256K context length. 🧵👇
88
180
60
1,256
461,149
One of our roles in LLM/VLM research at NVIDIA is to explore effective data recipes for training large-scale models and share them to the public—an area where transparency has been limited, as seen with models like Gemini, GPT-4o, Qwen-VL models etc. The Eagle2 project aligns closely with this mission. In this work, we have openly detailed our findings in curating the datasets to develop a frontier VLM model, and we’re glad to see that the community is finding these contributions valuable.
The Eagle 2 paper from Nvidia is such a goldmine.
1
17
1,639
The checkpoint for eagle2.5 has been released at huggingface.co/nvidia/Eagle2…
Eagle2.5 natively supports long-context without using any compression module. Eagle2.5-8B has: • got 6 out of 10 SOTA on long video benchmarks • beat GPT-4o (0806) on 3/5 video tasks • beat Gemini 1.5 Pro on 4/6 video tasks • got SOTA result on Hour-long video benchmark.
10
995
Guilin Liu retweeted
I did not notice this until just now. Thank you @andimarafioti for the recommendation! Very glad that even though Eagle 2 is not our latest work, people still find it very useful.
The Eagle 2 paper from Nvidia is such a goldmine.
1
3
1
19
6,130
The Eagle 2 paper from Nvidia is such a goldmine.
1
25
2
293
20,084
🥇Our NVIDIA Llama Nemotron Nano VL model is #1 on the OCRBench V2 leaderboard. Designed for advanced intelligent document processing and understanding, this model extracts diverse info from complex documents with precision, all on a single GPU. 📗 Get the technical details on the newest Nemotron model ➡️ nvda.ws/43L3e0a 📝 Try out the NVIDIA NIM ➡️ nvda.ws/4jyZ0yD
2
44
6
233
22,674
Cool paper from @nvidia Prior methods for training LLMs for tool use rely on imitation or distilled reasoning, limiting generalization. Nemotron-Research-Tool-N1 uses rule-based reinforcement learning. It trains models with binary rewards evaluating only tool call structure and correctness, enabling autonomous reasoning. 📌 Binary format and correct tool call reward teaches autonomous reasoning over imitation. 📌 Binary rule-based reward prevents reward hacking, boosting real-world generalization (80.38 percent Live BFCL). 📌 Using binary rewards on structure and tool call leverages SFT data without detailed reasoning steps. ---------- Methods Explored in this Paper 🔧: → The model uses a structured reasoning and action output format. → A binary reward checks adherence to this format and exact match of parsed tool calls to ground truth. → Training uses the Generalized Reinforcement Policy Optimization GRPO algorithm on processed datasets. → Nemotron-Research-Tool-N1-7B achieved 84.82 percent accuracy on BFCL and 81.28 percent on API-Bank, outperforming GPT-4o. ------------ Paper - arxiv .org/abs/2505.00024v1 Paper Title: "Nemotron-Research-Tool-N1: Tool-Using Language Models with Reinforced Reasoning"
4
45
2
191
15,605
Tool-using LLMs can learn to reason—without reasoning traces. 🔥 We present Nemotron-Research-Tool-N1, a family of tool-using reasoning LLMs trained entirely via rule-based reinforcement learning—no reasoning supervision, no distillation. 📄 Paper: arxiv.org/pdf/2505.00024 💻 Code: github.com/NVlabs/Tool-N1 (Please consider giving us a ⭐️ to stay updated on the upcoming code release!) 🧠 Why this matters: Existing tool-call models rely heavily on supervised reasoning traces from stronger models—costly, brittle, and often imitative. We ask: Can LLMs learn to reason directly from tool success signals? 📦 What we did: – Train Qwen2.5-7B/14B with simple binary reward on tool-call correctness + reasoning format in R1-style – No reasoning traces needed – Evaluate on BFCL, API-Bank, and ACEBench – Also study the role of SFT, RL, and widely adopted SFT-then-RL recipes in training Tool-Calling models. 📈 Key findings: – Tool-N1-7B/14B obviously outperform GPT-4o and open baselines on all benchmarks – Widely adopted SFT+RL paradigm doesn’t necessarily lead to better performance than Pure RL. – Binary reward > fine-grained reward, esp. for real-world queries – Scaling works: bigger = better gains under our RL setup 🌟 Takeaway: Reasoning doesn’t have to be taught. With just a binary signal, LLMs can learn to reason and act. Tool-N1 sets a new direction for scalable, supervision-light tool calling model training
2
93
5
354
40,790
Eagle2.5 natively supports long-context without using any compression module. Eagle2.5-8B has: • got 6 out of 10 SOTA on long video benchmarks • beat GPT-4o (0806) on 3/5 video tasks • beat Gemini 1.5 Pro on 4/6 video tasks • got SOTA result on Hour-long video benchmark.
Nvidia just dropped Eagle 2.5 Boosting Long-Context Post-Training for Frontier Vision-Language Models
2
2
1
7
2,015
Guilin Liu retweeted
Thank you AK! Excited to introduce Eagle 2.5, NVIDIA’s latest vision-language model that brings strong long-context capabilities across both image and video understanding — all with just 8B parameters. Most existing VLMs struggle with high-res inputs and long video contexts. Eagle 2.5 is designed to tackle both — supporting up to 512 video frames and trained jointly on image + video data. We introduce a new benchmark-scale dataset, Eagle-Video-110K, with over 110K annotated samples, including QA, localization, and summarization. Videos range from a few minutes to 3 hours — pushing the limits of long-form visual reasoning. Key techniques: • Information-First Sampling: spatially aware, quality-preserving frame selection • Mixed image-video training for generalization • Progressive long-context recipes up to 128K tokens • Optimized decoding and inference for efficient deployment Strong results across the board: • 6 out of 10 SOTA on long video benchmarks • Outperforms GPT-4o (0806) on 3/5 video tasks • Outperforms Gemini 1.5 Pro on 4/6 video tasks • Matches or beats Qwen2.5-VL-72B on multiple key datasets • Strong image understanding with consistent improvement over Eagle 2, matching Qwen2.5-VL. Evaluated on: • Video-MME • MVBench • Charades-STA • 1-Hour Video QA • EgoSchema • MLVU, LVBench, and more… These tasks stress-test long-form visual understanding with dense supervision and temporal reasoning. Model, demo, and dataset will be released soon. Explore the project here: nvlabs.github.io/EAGLE/ Code: github.com/NVlabs/EAGLE Tech Report: huggingface.co/papers/2504.1… We're excited to contribute toward long-context, general-purpose VLMs — and would love to hear your feedback or ideas for collaboration.
Nvidia presents Eagle 2.5! - A family of frontier VLMs for long-context multimodal learning - Eagle 2.5-8B matches the results of GPT-4o and Qwen2.5-VL-72B on long-video understanding
1
10
1
53
19,225
Guilin Liu retweeted
Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵
93
383
121
1,906
468,660
Guilin Liu retweeted
Mr. @pmddomingos This is a country whose leader blatantly says "We lied, we cheated, we stole… we had entire training courses." And thus there's conceited clown like you to spread China hate everywhere. Your self-imagined star-spangled awesomeness doesn't change the fact that Chinese researchers have become a major force in the AI community. China is also leading in industry general autonomy, robotics and AI applications. Your word can't change this fact and the successes don't come with fraud. If you think there’s a problem with this, there’s a problem with you.
Unbelievable. China has a huge problem with scientific fraud, and when someone alludes to it the conversation is about how racist they are.
8
10
2
213
31,263
Guilin Liu retweeted
Thank you AK! @_akhaliq This is just a beginning of a long journey, as we focused more on the model design space with multi-encoders, and fair comparisons under controlled settings. More will come in future versions! 🧵[1/n] Try our model & demo: GitHub: github.com/NVlabs/Eagle HuggingFace: huggingface.co/NVEagle Report: huggingface.co/papers/2408.1…
Nvidia presents Eagle Exploring The Design Space for Multimodal LLMs with Mixture of Encoders discuss: huggingface.co/papers/2408.1… The ability to accurately interpret complex visual information is a crucial topic of multimodal large language models (MLLMs). Recent work indicates that enhanced visual perception significantly reduces hallucinations and improves performance on resolution-sensitive tasks, such as optical character recognition and document analysis. A number of recent MLLMs achieve this goal using a mixture of vision encoders. Despite their success, there is a lack of systematic comparisons and detailed ablation studies addressing critical aspects, such as expert selection and the integration of multiple vision experts. This study provides an extensive exploration of the design space for MLLMs using a mixture of vision encoders and resolutions. Our findings reveal several underlying principles common to various existing strategies, leading to a streamlined yet effective design approach. We discover that simply concatenating visual tokens from a set of complementary vision encoders is as effective as more complex mixing architectures or strategies. We additionally introduce Pre-Alignment to bridge the gap between vision-focused encoders and language tokens, enhancing model coherence. The resulting family of MLLMs, Eagle, surpasses other leading open-source models on major MLLM benchmarks.
1
11
1
61
86,980