Trung Vu retweeted
Terminal-Bench is a leading benchmark for agents. Unfortunately it’s hard: most small coding agents get very low scores on TB2, so training/system ablations look flat - you can't tell what's working.
Announcing OpenThoughts-TBLite - 100 curated TB2-style tasks, difficulty-calibrated so even 8B models can make progress. It's designed to give researchers measurable signal during development, providing faster feedback for experimental iteration while closely tracking true TB2 performance🧵
Very excited for this integration. Will make agentic RL much much easier.
Releasing the official SkyRL + Harbor integration: a standardized way to train terminal-use agents with RL.
From the creators of Terminal-Bench, Harbor is a widely adopted framework for evaluating terminal-use agents on any task expressible as a Dockerfile + instruction + test script.
This integration extends it: the same tasks you evaluate on, you can now RL-train on.
Blog: novasky-ai.notion.site/skyrl…
🧵
5
913
cool research describing regimes under which LoRA *matches* full fine-tuning performance. also great to see our OpenThoughts dataset being used in the experiments!
LoRA makes fine-tuning more accessible, but it's unclear how it compares to full fine-tuning. We find that the performance often matches closely---more often than you might expect. In our latest Connectionism post, we share our experimental results and recommendations for LoRA.
thinkingmachines.ai/blog/lor…
4
373
Trung Vu retweeted
📢📢📢 Releasing OpenThinker3-1.5B, the top-performing SFT-only model at the 1B scale! 🚀
OpenThinker3-1.5B is a smaller version of our previous 7B model, trained on the same OpenThoughts3-1.2M dataset.
it’s hard to stress the degree to which the *US* open source ecosystem is increasingly built around Chinese open source even as US companies continue to advance the frontier. we need an American DeepSeek project.
My latest post: The American DeepSeek Project
Build fully open models in the US in the next two years to enable a flourishing, global scientific AI ecosystem to balance China's surge in open-source and an alternative to building products ontop of leading closed models.
3
479
the shift from “VC dimension too large” and “objective function is non-convex” to “gpus go brrr” and “yolo runs” is a Kuhnian paradigm shift.
the former seemed rigorous and serious; the latter naive and childish. the courage to be naive was imo what made Ilya&co special.
Ilya on deep learning in 2015
On vision and how to understand deep learning.
interconnects.ai/p/ilya-on-d…
2
8
1,004
some very interesting data on cross-benchmark correlation coming out of open thoughts!
We evaluated more than 1000 reasoning LLMs on 12 reasoning-focused benchmarks and made fascinating observations about cross-benchmark comparisons. You can explore all that data yourself on our HuggingFace spaces page. (1/4)
3
238
Trung Vu retweeted
Day 5: Humanity's Last Exam!
Insights/Fun facts:
1. Created by Scale and Center for AI Safety (which is co-founded by Dan Hendrycks of MMLU-fame).
2. Supposed to be incredibly hard for AI models (hence the name), but we all know how this is going to go.
3. There are 193 unique subjects based on 8 overall categories. 76% of the questions are exact match questions while 24% are multiple choice. The largest category of questions is math which accounts for 41% of the dataset.
4. There are 579 authors who have contributed questions to this dataset with Dmitry D contributing the most number of questions at 99. Try creating one and you will realize how hard it is!
Links below for you to drill down and explore!
Trung Vu retweeted
Day 2 of drilling down into popular benchmarks for models/agents.
Benchmark #2: GSM8K, (or Grade School Math 8K)
A dataset of 8,500 high-quality, linguistically diverse grade school math word problems.
Our viewer for GSM8K dataset reveals some pretty interesting insights:
ALT GSM8K Dataset Explorer
Understanding what’s in the data is a high leverage activity when it comes to training/evaluating models and agents.
This week we will drill down into a few popular benchmarks and share some custom viewers that will help pop up various insights.
Our viewer for GPQA (Google Proof QA) dataset reveals some pretty interesting insights
ALT GPQA Dataset Explorer
Trung Vu retweeted
Understanding what’s in the data is a high leverage activity when it comes to training/evaluating models and agents.
This week we will drill down into a few popular benchmarks and share some custom viewers that will help pop up various insights.
Our viewer for GPQA (Google Proof QA) dataset reveals some pretty interesting insights
ALT GPQA Dataset Explorer
Trung Vu retweeted
OpenThinker3-7B is the SOTA open-data 7B reasoning models, powered by our new OpenThoughts3-1.2M dataset! We beat DeepSeek-R1-Distill-7B on our benchmarks by 33% on average 🚀🚀.
Our new paper 📝 offers unique insights into data curation. Filtering questions works, but filtering answers does not. Synthetic questions are surprisingly effective, and sometimes weaker models are better teachers. Check out our new paper, thread, and blog for more details!
Announcing OpenThinker3-7B, the new SOTA open-data 7B reasoning model: improving over DeepSeek-R1-Distill-Qwen-7B by 33% on average over code, science, and math evals.
We also release our dataset, OpenThoughts3-1.2M, which is the best open reasoning dataset across all data scales. Full details are in our ✨new paper✨ - below we share the highlights:
BTW, it also works on non-Qwen models😉 (1/N)
Trung Vu retweeted
📢📢📢 Announcing OpenThoughts3!
We also release a paper this time ⬇️
Announcing OpenThinker3-7B, the new SOTA open-data 7B reasoning model: improving over DeepSeek-R1-Distill-Qwen-7B by 33% on average over code, science, and math evals.
We also release our dataset, OpenThoughts3-1.2M, which is the best open reasoning dataset across all data scales. Full details are in our ✨new paper✨ - below we share the highlights:
BTW, it also works on non-Qwen models😉 (1/N)
Trung Vu retweeted
I'm excited to announce what we have been working on for months. Announcing OpenThinker3, the strongest 7B reasoning model with open data. Also more than 1000 experiments on what works and what doesn't for post-training data curation.
Announcing OpenThinker3-7B, the new SOTA open-data 7B reasoning model: improving over DeepSeek-R1-Distill-Qwen-7B by 33% on average over code, science, and math evals.
We also release our dataset, OpenThoughts3-1.2M, which is the best open reasoning dataset across all data scales. Full details are in our ✨new paper✨ - below we share the highlights:
BTW, it also works on non-Qwen models😉 (1/N)
Trung Vu retweeted
OpenThinker3-7B now outperforms Nvidia’s Nemotron-Nano-8B and DeepSeek-R1-Distill-Qwen-7B on reasoning benchmarks. It’s the strongest open-data reasoning model at the 7B scale 🧠
Today we’re releasing the full data curation recipe, dataset and model along with our paper. Huge thanks to our incredible team who made it possible 🙌
It was a thrill to build this together 🚀
Announcing OpenThinker3-7B, the new SOTA open-data 7B reasoning model: improving over DeepSeek-R1-Distill-Qwen-7B by 33% on average over code, science, and math evals.
We also release our dataset, OpenThoughts3-1.2M, which is the best open reasoning dataset across all data scales. Full details are in our ✨new paper✨ - below we share the highlights:
BTW, it also works on non-Qwen models😉 (1/N)
we created a SOTA reasoning dataset. a massive effort and an extremely fun time working with an awesome team :)
Announcing OpenThinker3-7B, the new SOTA open-data 7B reasoning model: improving over DeepSeek-R1-Distill-Qwen-7B by 33% on average over code, science, and math evals.
We also release our dataset, OpenThoughts3-1.2M, which is the best open reasoning dataset across all data scales. Full details are in our ✨new paper✨ - below we share the highlights:
BTW, it also works on non-Qwen models😉 (1/N)
cool work from @leonardtang_ @qw3rtman and the crew at @haizelabs showing that smol models can do well with RL :)
You don’t need frontier lab resources for frontier lab automated LLM evaluation.
To prove this, we’re open-sourcing j1-nano and j1-micro: two absurdly tiny (600M & 1.7B parameters) but mighty reward models competitive with orders-of-magnitude larger peers.
j1-nano and j1-micro were trained in <1 day on a single A100 GPU.
1
7
702
awesome work and a very surprising finding from the team! really makes you rethink how rl training works.
🤯 We cracked RLVR with... Random Rewards?!
Training Qwen2.5-Math-7B with our Spurious Rewards improved MATH-500 by:
- Random rewards: +21%
- Incorrect rewards: +25%
- (FYI) Ground-truth rewards: + 28.8%
How could this even work⁉️ Here's why: 🧵
Blogpost: tinyurl.com/spurious-rewards
