CM retweeted
Tinker’s Dimuon is Muon with delayed diagonal conditioning, Polar Express and a rank-dependent update norm. I recovered its update rule through the API, borrowing @mrtnm’s approach to StableReinforce. Here’s how 🧵
Dimuon is a new optimizer we built for LoRA fine-tuning. In our SFT tests, it often reaches lower loss in fewer steps than AdamW. We're still experimenting with Dimuon; you can Tinker with it today and share your feedback!
See our optimizers guide at tinker-docs.thinkingmachines…
Hard agree, especially with the last point.
There is a lot more you can squeeze out from GPUs than people first imagine. Like a LOT more
Startup @flappyairplanes is betting the future of AI is making models more efficient. cnb.cx/3UiruGm
CM retweeted
dessa by @jack's company Block
incredibly impressive
best speech to speech model ive seen bar none
CM retweeted
new from us at @blocks
Meet Dessa, our AI model trained from scratch on nothing but speech.
We believe speech is the natural interface for technology. So we built Dessa.
And we’re inviting AI safety researchers to explore Dessa.
Learn more: block.xyz/dessa
CM retweeted
Meet Dessa, our AI model trained from scratch on nothing but speech.
We believe speech is the natural interface for technology. So we built Dessa.
And we’re inviting AI safety researchers to explore Dessa.
Learn more: block.xyz/dessa
It’s time to recalibrate whatever we knew about voice AI.
I’m excited to share what we’ve been working on at @blocks
This is the GPT3 moment for speech.
Meet Dessa, our AI model trained from scratch on nothing but speech.
We believe speech is the natural interface for technology. So we built Dessa.
And we’re inviting AI safety researchers to explore Dessa.
Learn more: block.xyz/dessa
RT @blocks: Meet Dessa, our speech model trained from scratch on human conversation.
Hear it for yourself: block.xyz/dessa http…
I will say, I’m in love with pre-training optimizers. The fact that you can only use first order derivative info, that your implementation must be distributed-friendly, that it must maintain sample efficiency under scaling
It produces so many beautiful ideas
If you order a waymo from Scarbs into downtown Toronto when it’s snowing and there’s salt on the road you get banned by anthropic for AI abuse btw
We’re bringing the magic of Waymo to Toronto! 🇨🇦
In the coming days, you’ll begin seeing Waymo’s all-electric vehicles driving through Toronto’s streets as we lay the groundwork for our ride-hailing service.
Our goal is to begin serving riders in 2028.
Sign up for future updates: waymo.com/waymo-in-canada
CM retweeted
We built an 8000 sqft. high-throughput wet-lab in 30 days, to solve molecular discovery - starting with pesticides.
Most teams in drug-discovery don’t receive target-organism verification until clinical trials. Pesticides allow us to rapidly validate our model in the target organism directly. With the help of our high-throughput lab, we were able to generate compounds that killed a $13 billion pest, the Fall Armyworm.
We believe that the bitter lesson is coming for biology.
True post-AGI company btw, you’re gonna need the best pesticides for when all white-collar work is solved and we have our own farms
We built an 8000 sqft. high-throughput wet-lab in 30 days, to solve molecular discovery - starting with pesticides.
Most teams in drug-discovery don’t receive target-organism verification until clinical trials. Pesticides allow us to rapidly validate our model in the target organism directly. With the help of our high-throughput lab, we were able to generate compounds that killed a $13 billion pest, the Fall Armyworm.
We believe that the bitter lesson is coming for biology.