
Member of Technical Stuff @flourishailabs | ex-Databricks Research, OG-MosaicML. I like it when brains inspire AI 🧠+🤖
NYC
Joined April 2021
- Tweets630
- Following2.4K
- Followers1.2K
- Likes5K
Pinned Tweet
I recently left @DbrxMosaicAI to join @flourishailabs.
Why did I decide to make the jump?
I believe that some of the most interesting problems in science might now be within reach. With all the wild advances in AI, there’s no better time to pursue the big questions.
Flourish is doing just that. We are a mix of AI researchers, engineers, and neuroscientists tackling one of the most ambitious problems in the field: human-level intelligence and learning with human-level efficiency.
I have immense respect for the founders @TRReardon , Robert Williams, @jovo, @_vishar and for the team - many of whom built CTRL-labs from the ground up.
I really loved my time at @databricks and MosaicML before that. The caliber of the people and the research is sky high, and I am incredibly proud of the work we did over the past 5 years. In particular I’d like to thank my mentors @jefrankle , @mcarbin and @bemikelive, as well as the cracked @DbrxMosaicAI team.
If you’re interested in working at the intersection of AI research and human intelligence, don’t hesitate to reach out.
Jacob Portes retweeted
Congrats to Reflection folks -- it's hard to get the first model out, and we hope you can rapidly accelerate contributions to the ecosystem from here :)
Introducing Beam: a highly efficient agentic open model with 501B total parameters and 23B active.
- Frontier reasoning efficiency
- Advances the Western open frontier on coding & agentic tasks
- Trained end-to-end from scratch
Full weights release this month.
Learn more about Beam: reflection.ai/beam
Jacob Portes retweeted
me: how do i open pdf
claude: You're right to ask. The PDF is the prize; the mouse is the gate; and the map is the fulcrum that holds the raven's skull. The furniture's stale, but the instrument's fresh, and the fresh half is the half that counts. I deleted your home directory.
contrastive models are so back. pleasantly surprised this works!
And only on 16B tokens!
Introducing Contrastive Language Model (CLM): an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions.
CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks.
With lightweight fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks.
We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed.
Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size.
📄 Blog: contrastive-lm.notion.site
💻 Code: github.com/Contrastive-LM/CL…
🗣️ Discord: discord.gg/5dAQEDJBs
🤗 Data & Models: huggingface.co/Contrastive-L…
More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵
Some caveats xitter.cf/ilobov/status/21030088…
If you are as confused as I was reading that 8b model sets SOTA on Deepswe and Terminal bench, this is for you: they use Opus/Fable to actually generate trajectories and CLM 8b picks the best actions based on those choices.
Honestly, I like verifiers research direction and this looks an impressive step towards better ones, but you have to tone down the announcements or make them clearer.
Jacob Portes retweeted
The gap between major model releases keeps shrinking.
2023: once every 73 days
2026 (so far): once every 18 days 🤯
Jacob Portes retweeted
I tried @typesafeai’s Jev on a few model-routing tasks and was really impressed by its speed: 95% lower latency than GPT-5.6 Luna, at 5× lower cost. It is fast enough to run inline during an agent session, monitoring every turn and dynamically switching models when needed. Such latency unlocks a new class of real-time, adaptive agent experiences!
Jacob Portes retweeted
sharing a new paper!
Augmented Lagrangian Predictive Coding
I think one of the coolest unsolved problems is how the brain does credit assignment.
thread 🧵
Introducing PC-ALM, a local-learning alternative to backpropagation.
Our method trains 1000-layer neural nets using only local dynamics, and without backprop.
Blog: pub.sakana.ai/pc-alm/
Standard deep learning relies on backpropagation. The brain, however, cannot implement backpropagation, at least not exactly. How can a physical system, such as the brain, solve multilayer credit assignment without explicit use of backprop?
We look for inspiration in two related fields: distributed optimization and NeuroAI.
In NeuroAI, predictive coding asks each neuron activation to solve an energy-based inference problem instead of using a standard forward pass. That inference step can be implemented as energy-minimization dynamics on local prediction errors.
This perspective -- each layer as a dynamical system -- has proven promising, but performance of predictive coding hasn't scaled well with depth. Credit signals at far ends of the network struggle to diffuse into internal layers.
We turn to distributed optimization, generalizing predictive coding to use an augmented Lagrangian instead of energy. This motivation stems back to a classic 1988 paper by LeCun, showing that the Lagrange multipliers of a deep network can be identified with gradients of a supervised loss. The augmented Lagrangian then bridges LeCun's perspective to the standard predictive coding that is used in NeuroAI.
We find that this new perspective yields a natural PC-like alternative to backpropagation, resulting in a method we call PC-ALM. PC-ALM differs from PC in that it introduces dual neurons (Lagrange multipliers) as part of the layer-local dynamics, resulting in each layer acting as a PI feedback control system to minimize local prediction errors.
We find that PC-ALM is capable of propagating signals to seemingly arbitrary depth, especially in deep narrow networks where standard PC struggles to learn.
Ultimately, our motivation here is to understand how distributed physical systems, such as the brain, can compute credit signals using only local coupling and local dynamics.
PC-ALM may also inform deep learning in neuromorphic hardware, where dynamics are cheaper than on GPUs.
Paper: arxiv.org/abs/2605.31022
Code: github.com/SakanaAI/pc-alm
words that claude has ruined for me:
- headline
- contract
- genuine
- bottleneck
- honest
- signal
- reframes
- everything
Jacob Portes retweeted
If you think about it, it's pretty embarrassing that frontier labs still pretrain models. Don't they know there's a theory for this? Given how big the model is, and how long you're going to train, you can predict the loss exactly. No need to actually train. They spend billions on this!
Here's the crux. I'm sick of the load bearing footguns that need to be fixed with seams and shims. This is itself the smell (or is it the signal?). I need to pump the breaks — step back from the slop — and reframe the whole session. No Claude Code, no Claude Cowork, no Claude.
Jacob Portes retweeted
🚀 Today we’re introducing 𝗔𝘂𝘁𝗼𝗜𝗻𝗱𝗲𝘅: Learning Representation Programs for Retrieval. 🚀
AutoIndex learns executable programs that determine how a corpus is represented to a search engine.
We think this points to a broader category of research: 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝘁𝗵𝗮𝘁 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝘁𝗵𝗲 𝗲𝘅𝗲𝗰𝘂𝘁𝗮𝗯𝗹𝗲 𝗽𝗿𝗼𝗴𝗿𝗮𝗺𝘀 𝗮𝗿𝗼𝘂𝗻𝗱 𝘁𝗵𝗲𝗺.
Jacob Portes retweeted
I had the honor of giving a keynote at the International Conference on Machine Learning in Seoul last week titled “What will be left for us to work on?” I addressed the widespread anxiety about how we should adapt as AI capabilities increase. I was thrilled by the talk’s reception, so I have made my slides available, annotated with a lightly edited transcript: cs.princeton.edu/~arvindn/ta…
I made three arguments. First, the "AI as Normal Technology" framework is a correct and useful as a way to think about AI’s impacts, unless and until there is some future discontinuity such as through recursive self-improvement. Second, even though we should take recursive self-improvement seriously, there is no milestone that companies might achieve in the lab that will suddenly put us all out of work. Third and finally, jobs of the future will be radically different, and a lot of adaptation will be needed. I shared my thinking about what this might look like and ended with a vision of human/AI “co-superintelligence”.
Jacob Portes retweeted
my paper won an award at icml 😁
some thoughts:
• this work was rejected from NeurIPS. i cleaned it up a small amount and it got great reviews from ICML! don't give up
• ICML received 24k submissions and only gives out 7 awards, which is crazy. feeling grateful
• i distinctly remember sitting at my desk two winters ago wondering if i would ever finish this project. most of all this is the product of sitting down and forcing myself to keep working for several months straight. the results emerged from running the experiments over and over and fixing a long sequence of tiny details. eventually, the curves looked like that 👇
• also happy that the insights in this paper are becoming more widely accepted: 3.3 bits/param, thinking about capacity "LLM as flashdrive" mentality
• the method here is used successfully for selecting midtraining data at least one frontier lab, which is cool!
• i am grateful to my collaborators, but Meta is no longer a great place for academic research imo and this almost never got published for a number of reasons. i shall not elaborate further
• for future work, i think analyzing the implications of on-policy algorithms on capacity, as well as LoRA and things like it, are fruitful potential research directions
• sadly i'm not in Korea but am following the conference online from california and happy to chat!
a nice end to one phase of my research career :)
I got nerdsniped by this recent paper from @ZyphraAI last week arxiv.org/abs/2606.24752.
As you continue (pre)training, does the ability to learn eventually decrease?
My takeaways in the thread 👇🧵
This dovetails nicely with this paper
Overtrained language models are harder to fine-tune (Springer et al. 2025) arxiv.org/abs/2503.19206
Provided evidence that the "effects of plasticity loss may already be observable in 1B models, finding that these models had lower finetuning performance after pretraining for longer than 2T tokens."
The Zyphra paper doesn't have a good explanation for “why”.
Some remaining questions:
- If the relationship between plasticity loss and model size is sublinear, are there better strategies to mitigate this (than just increasing model size)?
- The authors didn't do any post training - it would be nice to see their results map directly to post training (maybe they'll open source the checkpoints?)
- How does this relate to the "grokking"/generalization debate? (e.g. How much do language models memorize? arxiv.org/abs/2505.24832 from @jxmnop)























