
Founder @SuperseedXYZ • AI orchestrator • ASI is coming soon
Joined January 2010
- Tweets16K
- Following428
- Followers15.8K
- Likes34.2K
Pinned Tweet
An alien von Neumann probe lands on Mars and doubles every two days.
We have an AI that might learn to deal with it. We’ve forbidden it from improving itself, because nobody can verify what it wants.
Fable 5.2 and me wrote Daylight 2029, a scenario about that choice.
Is Google back in the game?
Neon seems on par with Opus 5.5 based on scores, maybe slightly ahead on some benchmarks and slightly behind on coding.
That means they’re only ~2 months behind Anthropic and might be catching up.
Introducing Gemini 4 Neon! Pushing the frontier across domains and punching above its model class in a meaningful way. To me, Neon feels like the right balance of intelligence, speed, and usability. Available now across @GoogleAIStudio, the API, and more!
2
7
422
This is funny, but likely true.
Many Anthropic employees probably want to invest in Situational Awareness 2.0 and will soon sell some of their shares to public investors that have liquidated their stocks in other companies to invest in the $ANT IPO which in turn caused massive losses to Situational Awareness 1.0 which was leveraged long those stocks.
The reason these guys with early access are saying that the models are bad is because they themselves are becoming like saturated benchmarks.
The models are great, but this crop of reviewers aren’t useful anymore.
David Iach retweeted
GPT-6.2 Astra scores 174 on the Epoch Capabilities Index (ECI), on par with Fable 5.2 and joining it above what the pre-Astra trend for reasoning models would have predicted.
This is based on testing we did prior to launch using only our own benchmarks. The ECI is continually updated as more benchmark results become available, and so these values may change in the future.
First impression on Astra 6.2 is a very good one. 😃
If this is the model that's good at math, a lot of breakthroughs will happen in the next few weeks.
Big!
Thrilled to announce $5B in funding for @IsomorphicLabs to accelerate our mission to Solve All Disease. We continue to make rapid progress reimagining drug design from first principles with AI to radically cut the time & cost to find new medicines. Gradually, then all at once...
1
201
Lots of gems in this episode.
That said I find the takeover scenario exposed by @paulfchristiano to be quite poorly reasoned.
I’ll rewatch that part just to make sure I’m not missing anything and post something later about it.
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario).
He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails.
He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it.
And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion.
Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
PSA
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario).
He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails.
He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it.
And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion.
Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
1
431
Really interested in this episode. What updated me most towards very short timelines was the progress in math.
As soon as an AI model got better at math than humans, it instantly solved thousands of super hard problems (a decade of progress in one month).
I now fear that whoever trains a model to be better than humans at AI research, could see similar results, where it doesn’t just lead to acceleration, but to a near-instant, extreme jump in capabilities.
I also believe multiple labs will get to the point where they’ll be able to start this type of RSI very soon.
Generally I’m an accelerationist, but lately I’ve become more cautious and I think that pacing is required.
Episode dropping tomorrow with @paulfchristiano.
He thinks there’s a 1 in 3 chance of extreme superintelligence within a year (absent any slowing down).
And he'll explain what labs and governments should do right now given that possibility.
Told Fable 5.2 to turn Daylight2029 into a music video and let's just say that I'm super impressed.
Very creative direction. Told it to learn from @anabology work.
Enjoy!
Here's Daylight2029
xitter.gq/davidiach/status/21212…
An alien von Neumann probe lands on Mars and doubles every two days.
We have an AI that might learn to deal with it. We’ve forbidden it from improving itself, because nobody can verify what it wants.
Fable 5.2 and me wrote Daylight 2029, a scenario about that choice.
1
86
Interesting.
This slide from the Anthropic IPO roadshow was a big factor in my decision to leave. It’s an insane plot and I’m surprised they made it public.
It shows the compute efficiency of pretraining over time. The y-axis is a **log scale**: they are seeing continued superexponential algorithmic progress.
Mythos was not a one-time deviation, but a point on a superexponential. They could already retrain Mythos with a fraction of the compute originally needed.
It doesn’t strictly imply a software-only intelligence explosion- they have been spending more on R+D. But algorithmic improvements do not require ridiculous creative genius and are on the verge of being automated. You do the math.
2
602
Looks like Dwarkesh updated as well.
3
378
ASI means most capital dissolves and becomes worthless. Maybe all of it. People mistakenly think that only labor is cooked.
A fab that builds 70' era chips wouldn't be worth much today, at best the cost of the materials.
Everything is obsolete post ASI, your IP, your tools and factories, nothing survives.
You won't frontrun the ASI by investing in whatever.
But the more bullish you are on AI the less the last point matters. AGi/ASI will render nvidia superfluous long before it has time to decline or become a zombie firm
1
4
391



















