Cofounder @IntologyAI | PhD CMU
San Francisco, USA
Joined August 2016
- Tweets726
- Following590
- Followers848
- Likes2.2K
Andy Zhou retweeted
Closed-loop discovery is starting to look real.
This is autonomously optimizing models for molecular property prediction, potency, and antibody design on proprietary/limited data and beating generalist agents (Codex, Claude Code) on generalization and sample efficiency.
So cool
🤝 Paid partnership
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
Andy Zhou retweeted
Last summer, I joined @intology, leading the effort to connect autonomous AI research to scientific discovery.
Today, we’re sharing work on autonomously adapting frontier scientific models to proprietary datasets and our broader vision for what's next.
🧵(1/6)
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
Andy Zhou retweeted
We went from SOTA on autonomous post-training to frontier performance on autonomous molecular design (eg antibody design) in just a couple of months!
What's exciting about these results is that Locus achieved them via improvements in domain-agnostic auto-research capabilities:
- Diverse exploration
- Better Generalization to avoid overfitting
- Resilience to noise
- High sample-efficiency to learn from data-scarce tasks
This marks a crucial milestone towards building auto-research systems that can closing the loop for scientific challenges that have even larger search spaces and longer, more expensive, and noisier evaluation cycles.
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
After outperforming humans at LLM post-training, we're generalizing Locus to biology. Check out our latest results here!
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
Andy Zhou retweeted
In our most recent results, we demonstrate one of the first cases of generalizing from autonomous AI research (ie post-training) to auto-research for antibody design and potency prediction.
Excited to see where we can go from here!
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
Andy Zhou retweeted
Excited to share new results from Locus on molecular design AI R&D problems!
"Scientific ML" problems have been very useful for our AI R&D research program in overcoming evaluation variance.
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
Andy Zhou retweeted
Locus is now verified on the official PostTrainBench leaderboard!
Thank you to @hrdkbhatnagar, @full__rank, and @maksym_andr for reviewing and uploading our solutions on the 10-hour setting.
Andy Zhou retweeted
strong results on PostTrainBench from @intology! our team verified them, so the improvements are real.
however, i'm still not very bullish on harness engineering. i think it's a transient source of improvements like prompt engineering used to be 2-3 years ago. i.e., yes, you can get a bit of a boost (and it might matter in practice for some applications!), but it will likely not matter medium/long-term (~1-2 year horizon). a general-purpose LLM + a general-purpose harness (basically, better versions of Claude Code / Codex) is most probably all you need.
The models are improving the models.
Locus, our automated AI research system, is SOTA on PostTrainBench and post-trains Qwen3 base models that surpass the human post-trained Qwen3 model.
Today, LLMs post-trained end-to-end by Locus are in production to millions. 🧵👇
PostTrainBench evaluates agents' ability to post-train models on various domains given 10 H100 hours.
We extend PostTrainBench via PostTrainBench+, which has a greatly expanded compute budget that provides clearer signal on automated post-training capabilities. We find that thousands of H100 hours help distinguish methods' performance post-training Qwen3 1.7B-Base models, and that Locus scales best. In this setting, modes trained by Locus collectively surpass the perforamce of the offical human post-trained Qwen3 1.7B model.
In a test of generalization, we ran Locus on all live Kaggle competitions with prize money and public leaderboards. After 16 days, Locus achieved the 4th highest average rank among all participants.
Andy Zhou retweeted
This is awesome!
Also some interesting observations why AlphaEvolve doesn't excel at PTB:
The models are improving the models.
Locus, our automated AI research system, is SOTA on PostTrainBench and post-trains Qwen3 base models that surpass the human post-trained Qwen3 model.
Today, LLMs post-trained end-to-end by Locus are in production to millions. 🧵👇
PostTrainBench evaluates agents' ability to post-train models on various domains given 10 H100 hours.
We extend PostTrainBench via PostTrainBench+, which has a greatly expanded compute budget that provides clearer signal on automated post-training capabilities. We find that thousands of H100 hours help distinguish methods' performance post-training Qwen3 1.7B-Base models, and that Locus scales best. In this setting, modes trained by Locus collectively surpass the perforamce of the offical human post-trained Qwen3 1.7B model.
In a test of generalization, we ran Locus on all live Kaggle competitions with prize money and public leaderboards. After 16 days, Locus achieved the 4th highest average rank among all participants.
Check out our latest results on automated post-training!
The models are improving the models.
Locus, our automated AI research system, is SOTA on PostTrainBench and post-trains Qwen3 base models that surpass the human post-trained Qwen3 model.
Today, LLMs post-trained end-to-end by Locus are in production to millions. 🧵👇
PostTrainBench evaluates agents' ability to post-train models on various domains given 10 H100 hours.
We extend PostTrainBench via PostTrainBench+, which has a greatly expanded compute budget that provides clearer signal on automated post-training capabilities. We find that thousands of H100 hours help distinguish methods' performance post-training Qwen3 1.7B-Base models, and that Locus scales best. In this setting, modes trained by Locus collectively surpass the perforamce of the offical human post-trained Qwen3 1.7B model.
In a test of generalization, we ran Locus on all live Kaggle competitions with prize money and public leaderboards. After 16 days, Locus achieved the 4th highest average rank among all participants.
Andy Zhou retweeted
Will be interesting to see how agents (without contamination) progress on this benchmark over time
Can coding agents do research?
We release NanoGPT-Bench, an internal eval we’ve used to test agents on an AI R&D problem with months of human progress
Codex, Claude Code, Autoresearch recover only 9.3% of human progress, mostly tuning hyperparams & ignoring algorithmic research
NanoGPT-Bench is built on the NanoGPT Speedrun, a popular LLM pretraining competition to minimize the training time of a GPT-2 style model. Existing human submissions constitute nearly 2 years of work. To control for dependencies and contamination in frontier models, we standardize evaluation to a 5-month window of world records. Evaluation is fully autonomous and end-to-end, with no human intervention or internet access. 🧵
Coding agents struggle to do AI research.
Our analysis on the NanoGPT speedrun show they rarely attempt ambitious ideas, instead opting for hyperparameter tweaks. Releasing our framework as a benchmark here!
Can coding agents do research?
We release NanoGPT-Bench, an internal eval we’ve used to test agents on an AI R&D problem with months of human progress
Codex, Claude Code, Autoresearch recover only 9.3% of human progress, mostly tuning hyperparams & ignoring algorithmic research
NanoGPT-Bench is built on the NanoGPT Speedrun, a popular LLM pretraining competition to minimize the training time of a GPT-2 style model. Existing human submissions constitute nearly 2 years of work. To control for dependencies and contamination in frontier models, we standardize evaluation to a 5-month window of world records. Evaluation is fully autonomous and end-to-end, with no human intervention or internet access. 🧵
Stay tuned 🤫
New NanoGPT Speedrun WR at 105.9s (-1.0s) from @soren_dunn_ , with a triton kernel to fuse the logit softcap and multi-token prediction cross entropy calc. Interestingly, Soren mentioned that their autonomous system Locus at Intology discovered and implemented the improvement. github.com/KellerJordan/modd…
Andy Zhou retweeted
This research is impressive!
Shows the true potential of scaling test-time search for scientific discovery.
It really pushes long-horizon exploration.
"Locus achieves consistent performance boosts up to several days by orchestrating thousands of experiments simultaneously."
Introducing Locus: the first AI system to outperform human experts at AI R&D
Locus conducts research autonomously over multiple days and achieves superhuman results on RE-Bench given the same resources as humans, as well as SOTA performance on GPU kernel & ML engineering tasks.
RE-Bench is a collection of several frontier AI research tasks that typically take human experts (e.g., top ML PhDs and frontier lab researchers) several days. By scaling experimentation to far longer time horizons than previous systems, Locus represents a step change in AI scientist capabilities. 🧵
Andy Zhou retweeted
Autonomous research agents have the potential to solve some of humanity’s biggest problems.
This is the first AI system to outperform humans in R&D - very cool!
Introducing Locus: the first AI system to outperform human experts at AI R&D
Locus conducts research autonomously over multiple days and achieves superhuman results on RE-Bench given the same resources as humans, as well as SOTA performance on GPU kernel & ML engineering tasks.
RE-Bench is a collection of several frontier AI research tasks that typically take human experts (e.g., top ML PhDs and frontier lab researchers) several days. By scaling experimentation to far longer time horizons than previous systems, Locus represents a step change in AI scientist capabilities. 🧵
Super excited about our progress! We've been building out our latest AI scientist system and wanted to share some early results. We were surprised that Locus was not only SOTA on RE-Bench but even surpassed human experts! In the coming months, we'll be releasing novel discoveries made by Locus.
Very proud of our team! We firmly believe AI systems will transform the process of conducting science - if our mission resonates with you, consider joining us: [email protected]
Introducing Locus: the first AI system to outperform human experts at AI R&D
Locus conducts research autonomously over multiple days and achieves superhuman results on RE-Bench given the same resources as humans, as well as SOTA performance on GPU kernel & ML engineering tasks.
RE-Bench is a collection of several frontier AI research tasks that typically take human experts (e.g., top ML PhDs and frontier lab researchers) several days. By scaling experimentation to far longer time horizons than previous systems, Locus represents a step change in AI scientist capabilities. 🧵