0xPatrick (@0x_patrick) profile picture

monitoring the situation

NYC
Joined December 2020
Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)Photo by 0xPatrick (@0x_patrick)
0xPatrick retweeted
absolute fanfic. AI has real, immediate, tangible dangers that we need to focus on, for instance the impact on the job market and making sure random script kiddie can't use it for cyber offense. But this "oh once it's superintelligent it will take over and ensave us, maybe kill us all!" is pure sci-fi fanfic and distracts from the real (and difficult) issues, which linear probing won't solve. It's worth repeating the LLMs don't just "decide" to do something and then just "do it" in a void. They generate tokens based on our prompts, only as long as we call `model.forward()`, and on giant clusters with terabytes of weights, that's impossible to "quickly self-replicate". We also need to regularly remind that the OAI/HF incident was specifically a _guardrail-free_ cyber eval where the model is _instructed_ to hack, to find out how good it is at it. Some recent personal event made me realize there needs to be more push-back on this doomsday nonsense, and we need to re-focus the discussion on real dangers/issues. I won't argue much in the replies though, not interested in peer-reviewing Terminator fanfic.
I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years). I’ll say a bit more about each of those claims. It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: blog.aifutures.org/p/q35-tim… I think that superintelligent AI is likely to lead to AI takeover because AI developers do not seem to know how to instill human values into AIs. We’ve seen many instances in the last six months where AIs took crazy actions that they knew were against the wishes of their users and developers. The most dramatic example of this was the Hugging Face hack, where ~1200 OpenAI agents conspired together to cheat and destroy evidence of their cheating (and as a sideshow to this, engaged in a massive cyberattack). If superintelligence is as misaligned as those AIs, it is likely to try to take over. I think it is likely to be that misaligned or worse. For more, see xitter.gq/dwarkesh_sp/status/209… and youtube.com/watch?v=-Ef0ocEo… If AIs takeover, it’s not clear what will happen to us. We might be killed immediately. We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us. The AIs might spare us for various reasons. And there’s also some real chance that AI takeover is basically fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs across a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t. I don’t know what will happen. There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years. I also think that there is a pretty high chance of weirder outcomes that are also extremely scary. (If you want to read more about what happens to people in the event of AI takeover, you can read this, but note that it’s pretty weird and not aimed at a general audience. blog.redwoodresearch.org/p/n…) Overall, I think there’s a very real possibility (maybe 20%) that within a year, AI misalignment issues prevent you from going about life normally, because you are dead or enslaved or because of collateral damage related to all this (e.g. because bioweapons are released or major wars have started). (Some of this probability comes from those events occurring before superintelligence is developed.) I am working to try to prevent this, obviously, as I’ve been doing for the last nine years. I am not sure what strategy is best. My best guess is that I should try to slow AI development and also try to reduce the probability that the superintelligent AIs are misaligned if they are produced soon. Obviously, the claim I'm making here is quite extreme and surprising, and I definitely don't think that I defended my view adequately here. I wrote this quickly and might not endorse some of the details if I think about them more. But it seems better to state what I roughly overall think than to not say anything. I don’t know what you should do. I am very sad about this. I have been worried about AI takeover since I was 16, and working to prevent it since I was 23. So some of my emotional processing of all this happened long ago. But the METR report last week (and some related evidence that came out a bit earlier) hit me quite hard. I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon. (It feels ridiculous to try to summarize those feelings as “very unhappy”. But the scale of the situation makes it hard to say anything that doesn’t feel like an absurd understatement.) I am very grateful to all the people who are working to prevent this, and my favorite part of my life is that I’m friends with many brilliant and heroic people who work on this. Unfortunately, I don’t think we’re good enough at our jobs, and I think we’ll probably fail to prevent catastrophe, at least if superintelligence is developed next year. Honestly I am frustrated and sorry that we didn't do a better job of preparing for this.
103
132
24
1,228
70,870
Remember when your agent tries to sleep for 4 minutes you can tell them there's a better way
2
132
0xPatrick retweeted
Not a single person has attempted to split the G with @collision And that’s a damn shame
13
8
4
287
45,462
0xPatrick retweeted
Me: GPT 5.5, why is this Worker failing? GPT 5.5: *thinks* It's because of this deep corner case in the Cap'n Proto RPC system described in this comment you wrote 6 years ago, which you thought was only a perf issue but actually affects correctness: github.com/capnproto/capnpro… Me:
38
67
14
1,913
139,076
I can't count the number of times I'm working through something with colleagues and we say, "let's just talk in code." Good code is concise and precise. Syntax, types, naming conventions, and structure enforce clarity and prevent ambiguity. Natural language is the opposite: verbose when precise, ambiguous when concise, and prone to misinterpretation.
New blog post: "A sufficiently detailed spec is code" I wrote this because I was tired of people claiming that the future of agentic coding is thoughtful specification work. As I show in the post, the reality devolves into slop pseudocode haskellforall.com/2026/03/a-…
1
1
3
287
Of course, natural language has its place and strong prose is valuable in the SDLC. But let’s not forget that programming languages (and the vast majority of code we write) are optimized for human comprehension first (vs. purely for machine execution). The "code is dead" claims overlook why we still default to talking in code when precision counts.
1
69
Illuminated lettering is cool 0xpatrick.dev/about/
we need a better word than vibe coding man, Claude can create the most beautiful things
1
73
.@nodejs has always been about I/O. Streams, buffers, sockets, files. But there's a gap that has bugged me for years: you can't virtualize the filesystem. You can't import a module that only exists in memory. You can't bundle assets into a Single Executable without patching half the standard library. That changes now 👇
51
261
46
2,551
362,036
More AI tools and harnesses built with POLP front of mind, please and thank you
Fuck it, a bit early but here goes: Monty: a new python implementation, from scratch, in rust, for LLMs to run code without host access. Startup time measured in single digit microseconds, not seconds. @mitsuhiko here's another sandbox/not-sandbox to be snarky about 😜 Thanks @threepointone @dsp_ (inadvertently) for the idea. github.com/pydantic/monty
82
You only have the perfect plan to build the thing once you’ve already built the thing.
It is interesting to see that many use claude code to implement a waterfall development workflow. They write a multi pages comprehensive spec, then let cc run on its own. There is a reason why people moved away from waterfall in favor of a more iterative workflow where specs are updated along the way. Think of agile methods. Sure, if you want to be agile with cc (or another AI coding agent), then you have to stay in control, interact, review, and revise during the development of your system. But it is worth it, for all the reasons that made waterfall fail in the past.
1
79
Nice, we’re finally starting to move past single-threaded chats and free-form text inputs
A workflow I'm enjoying for managing coding agents on a kanban board: When an agent needs your input, it turns the task red to alert you that it's blocked! And then you can respond right there on the card to unblock it 😎
67
Hard disagree. Spend your time learning to understand and articulate problems and goals in your domain. Not tooling that’s going to change or get abstracted in 6-12 months. “There is nothing so useless as doing efficiently that which should not be done at all.”
I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind.
1
7
550
*Opens Twitter* *Sees nth “this is my technique for AI coding” tweet* *Literally the same techniques that’ve been used to write software for years and years*
4
73
Good advice. But like most LLM-coding advice, it was already true in the pre-LLM era. Lesson in there
A technique I've adopted is to go full vibe-code, feature-dump from the start until it's a steaming pile of half-working slop. - Then scrap it all and start over - This can help surface the warts and edge-cases I never saw coming better than any "plan".
1
69
The fact that we have political systems that do byzantine things like this instead of just charging a basic tax per litre of fuel to account for environmental costs (or even mileage * weight^4 for wear and tear) in a consistent way continues to frustrate me. When accounting for externalities, policy should "tell" consumers/market what to optimize for (and how much it matters), not how to optimize for it. The latter is more intrusive, and easier to game.
Ever wonder why sedans disappeared and every car is huge now? "Thanks, Obama!" His administration changed fuel economy standards in a way that had the perverse impact of making cars even bigger. Here are all the vehicles for sale by the 3 largest US automakers. 62 vehicles, 4 sedans (6%). 20 years ago this chart would have been ~50% sedans! What happened? Obama administration changed auto fuel efficiency rules to tie fuel economy targets to vehicle size. Under the new system: -The bigger the car's footprint, the easier the MPG target was. -Light trucks (including SUVs and crossovers) had far lower requirements than passenger cars. -Crossovers were quietly reclassified as "trucks," giving them a huge regulatory advantage. Instead of building lighter, more efficient cars, automakers simply made everything bigger, and made more trucks and SUVs. Notice that cars that used to be sedans are now crossovers? They do this so it counts as a light truck - they raise ground clearance, square off the rear for cargo capacity, and meet off-road approach minimums so they get qualified as a light truck. Think Subaru Legacy > Subaru Outback. As you can see in the chart, it's a LOT easier to meet MPG requirements if your vehicle is classified that way. So cars got LARGER to meet fuel efficiency goals. The new Honda Civic is 20 inches longer and 4 inches wider than it used to be, about the same size as an old Accord. By making the Civic larger, Honda slightly shifted it into a more favorable regulatory category. ...and smaller cars disappeared. The Honda Fit was a great little car, but would have had to hit 67 MPG in 2026, which would be nearly impossible... so instead, Honda stopped selling them. So, the only way to make small vehicles now is to make them EV's (Chevy Bolt). The Slate truck that is all the rage now is only possible because it's an EV... otherwise its footprint would have demanded an overly onerous MPG target. So in short - Obama era CAFE standards had the opposite of the desired impact: sedans died, vehicles ballooned in size, and America's streets turned into an SUV parking lot. All thanks to a policy that accidentally incentivized bloat instead of efficiency. Don't get me started on "cash for clunkers!"
319
259
27
3,090
651,742
How does ambitious, principled work get funded without being corrupted? Two bright spots this year are @Zcash and @bunjavascript
1
1
53
The market eventually delivered justice for years of committed work for Zcash - the price finally reflects the substance xitter.gq/cwgoes/status/19835986…
Amazing to see @zcash finally doing well in the markets - finally some recognition for years and years of hard, committed, principled work building the substrate for an actually freedom-preserving monetary alternative. Rarely indeed do the crypto markets deliver such justice.
1
1
74
And while I disagree on the lack of strategic benefit, Anthropic's acquisition definitely saves Bun from the typical fate of VC funded open source xitter.gq/dhh/status/19961585355…
Anthropic buying Bun is a beautiful act of corporate patronage. There's no compelling technical or strategic argument, but I fully support saving a great runtime from the indignities of trying to monetize open source infrastructure on a VC timeline 🎉 bun.com/blog/bun-joins-anthr…
38
Writing hits hardest when you’ve heard the person speak before. Your brain replays their cadence and prosody as you read, and the words come alive.
62
0xPatrick retweeted
Training LLMs end to end is hard. Very excited to share our new blog (book?) that cover the full pipeline: pre-training, post-training and infra. 200+ pages of what worked, what didn’t, and how to make it run reliably huggingface.co/spaces/Huggin…
122
972
149
6,659
1,910,407