Editor + founder @ReadTransformer, covering AI policy. Prev: @TheEconomist, @Protocol, @finimize

London
Joined October 2009
As I said on @LBC: AI companies can no longer reliably control their systems.
4
5
36
4,903
just fell to my knees
We've decided to sunset Arc on December 22, 2026. That's not an easy sentence to write. We spent years building Arc and celebrating each and every release along the way: we have so much pride and love for what we made alongside the Arc community. Our priority is to deliver best-in-class security for our members. Keeping a browser secure takes constant, dedicated work that is only increasing in the age of AI. Dia was built from the ground up to meet today’s security standards, so that's where our engineering and security teams will focus moving forward. We've worked to make switching to @diabrowser as easy as possible for Arc members. Here's how: · Your passwords, bookmarks, extensions and history come over to Dia seamlessly. · Spaces, Folders, and Tab Groups are in Dia and import over with your data intact. · Dia now surpasses Arc benchmarks in stability, speed, and security. · The browser features you use in Arc are free in Dia, and the AI is optional. Thank you for using Arc, for pushing us to make it better, and for building alongside us. The team that made Arc is the same team behind Dia, and some of our most ambitious work is just ahead. We'd love for you to be there with us when it lands.
1
108
Shakeel retweeted
This summer, we invited METR and Redwood Research to pilot an embedded evaluation of our internal monitoring system, providing deep access to our infrastructure over three weeks. Their findings helped strengthen safeguards as we expanded the system and will help shape future research collaborations with METR and other third party evaluators.
Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.
138
64
30
1,072
109,849
Stop everything you’re doing and watch this bizarre AI-generated Eurovision-style music video from Nvidia’s GTC conference
ZERO mentioned of sovereignty on stage yesterday from Jensen Huang at GTC Berlin. The new pitch is *AI will make you rich* Interesting to see how this lands with the Europeans…
2
344
ZERO mentioned of sovereignty on stage yesterday from Jensen Huang at GTC Berlin. The new pitch is *AI will make you rich* Interesting to see how this lands with the Europeans…
2
1
2
444
The main strategy Claude came up with was to exploit existing frustrations about the cost of living to discourage people from voting with a message that “neither party cares, the system is broken, why bother.” transformernews.ai/p/claude-…
1
1
2
214
👀
Dario — Thank you for taking the time to respond to my previous post. I think it's great when this kind of engagement happens in public rather than on company Slacks, and I hope to see more of it. I agree that Anthropic isn't best modeled as a profit-maximizer. Your post gave several compelling examples of Anthropic taking competitive hits for safety, and insofar as my previous post suggested otherwise, that's on me. But I don't think I made my main point as clearly as I could have, so let me try again. It comes down to four key claims: 1. Anthropic seems to be racing at breakneck speed to build recursively self-improving superintelligence (RSI). By its own lights, that could kill billions of people or literally destroy the world. 2. Risking destroying the world is very hard to justify. It is only conceivably acceptable if serious attempts to put RSI under robust societal governance are failing. 3. Far from meeting that burden, Anthropic, when push comes to shove, often works to keep governments from tying its hands. It seems to care much more about preventing government overreach than about reducing the risk that it and other AI companies will destroy the world. 4. Even if the AEGIS Act really was an honest mix-up, as you suggest, that is itself a serious problem. It suggests Anthropic isn't treating the governance of this technology with anything close to the gravity it deserves. In greater detail for those who want it: 1. Anthropic is racing to RSI, and by its own lights that is insanely risky. Launching RSI could literally destroy the world. That's according to you, your co-founders, and most of your safety staff and capabilities researchers, not to mention many AI researchers outside your walls. You are intentionally trying to create generally intelligent systems more capable than all humans that you claim in six to twelve months “could be capable” of taking over the entire internet before we have the ability to reliably monitor, control, or align them. “Succeeding” at RSI could kill or permanently disempower billions of people in the near future, including me and everyone I love. Yet Anthropic’s primary focus seems to be marshalling an immense amount of resources and talent in a race to get there first. 2. Risking destroying the world carries an enormous burden of justification. I believe the vast majority of people do not want Anthropic to do this (to the extent they know what it's doing). Imposing even a modest chance of catastrophe on billions of people who never consented requires an extraordinary justification. The central one Anthropic has offered: someone will build RSI regardless, so it's better for a safety-focused lab to be at the front of the pack, using that position to support strong AI governance. (Other reasons Anthropic has given, like modestly accelerating improvements in healthcare or economic growth, don’t seem remotely proportionate to the risks Anthropic is imposing by its own lights.) But that argument only works as a last resort. Racing to RSI is only conceivably okay if attempts to institute robust societal governance to manage that race are failing despite your best efforts. A lab that relies on this justification should be a strong, consistent supporter of efforts to enact governance that could bind it, including governance that could slow it and significantly constrain what it does. 3. Rather than supporting efforts at strong regulation, Anthropic has intervened to keep governments from tying its hands. Anthropic has sometimes helped: it played a very helpful role in passing transparency bills in Illinois and ongoing efforts in Massachusetts, and I'm sincerely appreciative. But transparency is far from enough for a technology that will be the world's most powerful superweapon at best, and an uncontrollable swarm of lethally superintelligent agents at worst. When it matters most, Anthropic has often pushed for weak regulation. The AEGIS Act is the most obvious example. No highly risky industry is regulated as anemically as AEGIS would regulate frontier AI: not nuclear, not banking, not aviation, not food safety, not the manufacture of explosives. In all of those industries, specialist regulators can address unsafe practices well before they cause harm. AEGIS would bar the federal government from intervening to block unsafe development or deployment of models until there is an "imminent risk" of catastrophic harm, and would require it to meet a demanding burden of proof in court. Worse, AEGIS would also prevent states from picking up the slack. This design makes perfect sense if you're a company trying to minimize the risk that the government can overly constrain you. It makes little sense if you want American democracy to govern frontier AI effectively. This isn't the first time. The initial version of California's SB 1047 would have let the Attorney General sue to stop developers like Anthropic from imposing unreasonable risks of catastrophic harm. It would also have created a regulatory agency to help set safety standards. Anthropic objected, and helped get both provisions removed before ultimately offering some support for the bill (which I also genuinely appreciate). I’m not saying that Anthropic should have supported 1047 in its initial form; that’s a complex issue and I realize there were questions about whether earlier versions of the bill were less politically feasible. Still, in Anthropic's own words, it helped weaken the bill because "[t]here is a substantial risk that the bill and state agencies will simply be wrong about what is actually effective in preventing catastrophic risk, leading to ineffective and/or burdensome compliance requirements." The same goes for New York's RAISE Act, whose initial version also would have let the Attorney General sue to stop unreasonable risks of catastrophic harm. Anthropic credits itself as a supporter of RAISE, but it did not support the bill while it was in the legislature, including when it nearly died. After the bill passed the legislature, Anthropic publicly critiqued it, saying "Any state proposals should be narrowly focused on transparency and not overly prescriptive." Why does Anthropic consistently take the risk of clumsy government action far more seriously than its own repeated warnings that companies like Anthropic will literally destroy the world by racing to RSI? 4. Even if AEGIS was an honest mix-up, that is itself a serious problem. Your response frames AEGIS as an early draft and a miscommunication. Perhaps you would have strengthened it and narrowed its preemption. (That wouldn't explain Anthropic's interventions to weaken SB 1047 or RAISE, but leave that aside.) Suppose that's exactly right. A federal framework for governing RSI is one of the highest-stakes documents your company will touch. Your post admits Anthropic "could have handled [these matters] better," but its tone ("This kind of thing happens a lot in large organizations") doesn't reflect the gravity of this issue. Anthropic talks about the need for “operational excellence” in critical areas, and I have a hard time believing that a similarly cavalier attitude would be tolerated while Anthropic was negotiating a key compute deal or preparing its next large pre-training run. In short The justification for racing rests on Anthropic fighting for strong governance, and Anthropic's record on regulation points the other way. Together with its breakneck racing, that fits a much darker picture of the company than the one that seems to prevail among your employees. I hope that Anthropic reflects on this picture, supports stronger regulation, exercises extraordinary caution when advocating for weak regulation, and doubles down on its founding purpose of helping the world safely navigate transformative AI together, instead of prioritizing Anthropic’s ability to win a race that imposes intolerable risks on everyone alive.
1
6
1,264
The BBC's tech and AI coverage is (and has for a long time been) woefully bad. It's a real disservice to the public — quite negligent, given the BBC's outsized role in the UK media landscape.
This quoted post is unavailable.
1
1
24
754
No new donors to Leading the Future in the first two weeks of October, if you cared.
2
1
9
1,848
Shakeel retweeted
It looks like the threats worked
Replying to @MarcACaputo
Headline changed on one of the stories and correction appended at the top.. politico.com/news/2026/10/16…
3
12
1
41
8,322
It seems this could be fairly easily solved by having official well-scoped connectors with ChatGPT/Claude/Instinct/Muse etc., no?
"Major U.S. banks are detecting unauthorized third-party AI agents operating inside customer accounts, with some agents attempting to take actions that appear to be independent of user instructions, sources tell Axios." axios.com/2026/10/22/ai-agen…
6
466
Shakeel retweeted
every day they wake up and write 100 new auditing bills and then i clock into my shift at the auditing bill analysis factory
3
3
64
3,269
6-word horror story I experienced at the office today: "Ajeya Cotra seminar cancelled, work emergency"
3
34
5
396
9,721
Shakeel retweeted
so i've been putting together a list of things i’d need at home to survive a few days without power, internet, or basic services. you know, in case of earthquakes, digital or otherwise. claude’s plan includes a hamster. i don’t have a hamster. who is he. where is he sheltering
52
33
9
1,262
30,567
i want this so bad
im surprised that no one talking about duet's to-do list! maybe its personal preference, but its the reason i keep reaching for it. for suggested tasks — this is much better than receiving a growth hacky chat message. to create tasks — you can simply brain dump them!
2
739
Shakeel retweeted
One fun way for journalists to use AI: Ask the strongest model to which you have access to do a deep dive on your work and find where it was cited or otherwise had an impact. You'll find things you had absolutely no idea about
1
5
2
16
3,256
Shakeel retweeted
🌹🌹DSA civil war over AI, @jdcapelouto reports semafor.com/article/10/21/20…
1
9
3
52
23,507
This is, obviously, extremely bad. It is also quite ironic given how nakedly Politico’s owner has tried to suck up to this admin — and completely failed.
Quite a letter from White House to Politico.
528
News! OpenAI's written to Gov. Gavin Newsom recommending a bunch of amendments to strengthen SB 53. Includes a bunch of stuff on AI R&D and shutdown controls.
12
33
5
147
22,633
My usual file uploading services aren’t working, here are screenshots of the full letter
1
6
498