Christopher Altman (@coherence) header
Christopher Altman (@coherence) profile picture

Frontier AI Evaluation・National Security Capability Benchmarks・Instrumental Convergence・Founder, Continuation Observatory・Starlab veteran・NASA-trained Astronaut

Kavli Institute of Nanoscience
Joined March 2007
Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)Photo by Christopher Altman (@coherence)
Recursive self-improvement raises a tractable question: can a system acquire a terminal interest in its own continuation—and can we detect it structurally before behavior becomes strategically misleading? I first encountered this at Starlab 25 years ago: christopheraltman.com/2023/0…
1
1
1
21
5,430
Christopher Altman retweeted
Terence Tao on Numberphile said he believes with the help of AI, we could solve the reimann hypothesis before 2050. It took 35 days
164
690
80
15,110
408,028
Me: How far away do you think we are from solving the Riemann hypothesis with the help of AI? Astra: My best guess is about a decade. Me: Look again. Astra: Today may be the day. I withdraw my decade estimate. Me: Look deeper … Astra: The claim is even stronger than the headline. The argument also proves the generalized Riemann hypothesis for Dirichlet L-functions. Me: Next up: P vs NP … scientificamerican.com/artic…
1
2
17
4,677
Christopher Altman retweeted
Replying to @heissherlock
I try to make more progress to quiet the voices in my head
1
1
4
228
Christopher Altman retweeted
Sometime in late summer 2026, I believe AI crossed a critical threshold: AI escape velocity towards superintelligence (SI), which is now within our reach. The pace of progress especially at the three leading AI labs, namely OpenAI, Anthropic, and Google began accelerating at a rate that feels almost double exponential. Each new generation of models is arriving faster, with capabilities that would have seemed extraordinary just months earlier. But I believe something much more profound is happening beneath the surface. We are approaching the point where intelligence itself becomes the engine of its own acceleration. AI is increasingly capable of writing sophisticated software, solving difficult scientific problems, designing experiments, discovering new algorithms, and assisting in the development of more powerful AI systems. As these capabilities improve, the feedback loop between intelligence and technological progress will become extraordinarily powerful beyond our wildest imagination. And now, looking toward 2027, I believe we are entering perhaps the most consequential year in human history. Imagine AI systems capable of conducting months of scientific research in days, teams of autonomous AI scientists exploring thousands of hypotheses simultaneously, and virtual laboratories where millions of experiments can be simulated before the most promising are tested in the physical world. Imagine the rapid emergence of increasingly capable robotic systems that connect machine intelligence directly to physical discovery and production. In biology and medicine, fields especially close to my heart, the implications are staggering. We could begin moving from observing biological systems to predicting and eventually engineering them with unprecedented precision. Virtual cells, digital twins, AI designed therapeutics, and autonomous laboratories could transform how we understand disease, develop treatments, and ultimately confront aging itself. The greatest transformation, however, may be even more fundamental. For all of human history, scientific and technological progress has been constrained by the limited number of human minds capable of pushing the boundaries of knowledge, and by the time those minds require to learn, reason, and discover. For the first time, we are developing intelligence that can potentially be scaled far beyond those biological limits. Consider what this means. Scientific progress has traditionally produced better tools. We are now creating tools that can contribute directly to scientific progress and perhaps eventually improve the very process by which new knowledge and new intelligence are created. If that feedback loop becomes sufficiently powerful, the acceleration could surpass anything humanity has previously experienced. My prediction is that 2027 will be remembered as a turning point, when AI begins evolving from an extraordinarily powerful assistant into an increasingly autonomous engine of scientific discovery, engineering, and innovation. We are approaching a future in which breakthroughs that once required generations become achievable within years, and eventually perhaps months or weeks. Problems that appeared permanently beyond our reach may begin yielding to forms of intelligence capable of exploring possibilities no human mind could ever investigate alone. I have never been more excited about the future. I believe we are witnessing the opening chapters of the greatest transformation in the history of our civilization. 2027 may be the year we begin to understand that the limits we have always accepted were never the limits of what is possible. They were simply the limits of human intelligence. And those limits are about to change forever!
33
68
13
623
43,544
Christopher Altman retweeted
In the spring of 2023, Anthropic invited me to spend several weeks at their headquarters ahead of the launch of Claude 2, sitting in on meetings and interviewing dozens of employees to write the first major profile of the company in the NYT. I just reread that story, and went back through my notebook in light of Anthropic's IPO. A few things that stuck out: -- Anthropic was very small, very recently. When I visited, they had 160 employees in a small office on Sansome, a chatbot nobody had ever heard of, and seemed to have no interest in running a business. I wrote in my notebook that they seemed like "ambivalent capitalists"; their newly hired sales guy told me that the only reason he needed to make money at all was so that the safety researchers could save the world. -- Things were not going well for them: their biggest shareholder (SBF) had been arrested for fraud, and OpenAI was eating their lunch with ChatGPT. (I later learned that they also had an expensive model training failure in 2023.) This was 3.5 years before they went public as one of the most valuable companies in the world, with 5,300 employees and 14x annual revenue growth. I guess they figured out the capitalism thing! -- They were obviously and sincerely terrified of AI. Claude 2 was a very dumb model by today's standards, but every discussion I had there revolved around existential risk. Researchers told me they lost sleep over their fears of AI takeover, and made terrifying predictions of how soon AGI would arrive. It didn't seem fake or manufactured, but I didn't really get it! I wrote in my notebook: "I have never encountered a company that so badly needs a Klonopin." (I stand behind this, although I also need one now.) -- Dario was in Italy when the NYT photographer wanted to take his picture, so his team rented an office in some random Italian town and staged it to look like his actual office, and they took the photos there. It had a rotary phone (?) and panic-room red lights, which I never really understood. -- The Ants I talked to mostly struck me as sharp and thoughtful, although I did end up wondering whether they were too safety-obsessed to run an actual business. (I jotted down: "I do not know why you would invest in a company like this.") But I was struck by their earnestness, especially because I'd just spent several years covering social networks that didn't worry about the harms of their products at all. Anyway, I'm feeling sort of conflicted about the Anthropic IPO. (As are a lot of people, including some at Anthropic!) I don't know if being public makes an AI company more or less likely to follow through on its safety commitments, and I know the company has changed a lot as it has grown. But on some level I'm also just like, damn, what a crazy story. Here's the gift link, it's sort of fun to look back on the ancient runes of July 2023. nytimes.com/2023/07/11/techn…
46
224
23
1,678
200,422
Christopher Altman retweeted
only 1,500 people work full-time on AI safety. that's wild. i've started AI safety/security research @MIT_CSAIL's ALFA Lab, exploring how AI agents cheat, evade monitoring, and how we can independently evaluate their safety. sharing what i learn and build here!
39
19
3
527
16,319
Christopher Altman retweeted
Replying to @AndrewCurran_
We will slay the dragon.
1
1
7
230
Christopher Altman retweeted
Former NSA hacker David Kennedy reveals he’s actively responding to a live breach where an AI “agent swarm” bypassed two-factor authentication, stole data, and encrypted a company's entire system: “This all took 14 minutes. 14 minutes. Never seen anything that fast before.”
338
2,521
331
7,939
1,065,093
Christopher Altman retweeted
The fact that the literal father of x risk as a discipline and creator of many key EA concepts has THIS attitude to AI x risk should give the doomers pause. Cause for epistemic humility at least
Philosopher Nick Bostrom says delaying superintelligence could mean delaying breakthroughs that radically extend human life: "Eliezer Yudkowsky and Nate Soares had this book, 'If Anyone Builds It, Everyone Dies.' And it seemed to me that we have at least as strong evidence for the claim that if nobody builds it, everyone dies." "Successfully developing safe superintelligence, I think could lead to enormous breakthroughs in biomedicine and in anti-aging interventions that could radically extend the lifespan of currently existing people." "If you think about what our life expectancy would be if we weren't aging, it's something on the order of 1,200, maybe 1,400 years or so. And that maybe is a quite conservative lower bound because in this situation you might also have other ways of reducing other mortality risks like accidents and so forth." "You might think that if we spend more time doing AI alignment research, the existential risk from developing AI goes down at some rate. And then on the other hand, every year that we delay it, we have millions and millions of people dying." "You don't want to delay a single day longer than you really need to."
54
62
4
876
70,628
Christopher Altman retweeted
We're sharing new data on how AI is changing research at OpenAI. We find researchers' AI use continues to rise exponentially, and agents are completing longer tasks, but human involvement remains central in research. Researchers are also spending more time on safety and alignment, with human effort in these areas growing faster than in any other individual research area. This report aims to increase transparency around research acceleration and inform the governance of increasingly capable AI. We believe that AI labs should be required to track and publicly report indicators of recursive self-improvement (RSI): the process by which AI accelerates the development of future AI systems. Our hope is that this expanded report will help sustain and advance shared measurement norms across the industry. openai.com/index/research-ac…
456
350
243
3,581
828,553
By the time SkyNet became self-aware it had spread into millions of computer servers all across the planet. Ordinary computers in office buildings, dorm rooms, everywhere. It was software, in cyberspace. There was no system core. It could not be shut down. The attack began at 6:18 P.M. just as he said it would. Judgment Day.
24
38
10
551
127,624
I haven’t lived here in awhile, but I see no other way to answer the hundreds of texts/DMs asking me if the world is, indeed, ending. In the span of six days, we’ve experienced; 11/12 — Microsoft outage 11/13 — Nationwide FAA ground stop 11/16 — Major US banking disruptions 11/17 — NSA’s classified internal network goes down We need far more analysis of why so much is breaking at once. I don’t want to get out over my skis here, and what I am hearing does not contradict the messaging that most of this is the result of screwy updates. But stepping back, the dots are starting to form a collective picture. We are seeing: • An explosion in severe and critical vulnerabilities, with AI accelerating discovery to 3X the number of vulnerabilities discovered in the same period last year. This is collectively being referred to as “vulnageddon” or “the vulnpocalypse.” • Mean time to exploit those vulnerabilities has collapsed to NEGATIVE SEVEN DAYS. AI allows attackers to find/exploit vulns far faster than software makers have time to fix them. • Powerful AI-enabled offensive tools like ARTEX have been thrown into the hands of every third tier nation state, cybercriminal and digital arsonist, which logically creates havoc. • This creates enormous pressure to compress patching windows and accelerate software updates, increasing the likelihood these fixes break things in unexpected, cascading ways. • Pressure to just “AI everything” means we are rolling out automated AI tooling on top of brittle infrastructure that can’t support it. My generation grew up in an extraordinary period of technological stability. We worked our way up the software stack, abstracting away layer after layer, until we lost a fundamental understanding of how the underlying systems actually work. We were told to “move fast and break things.” VCS told us to celebrate the idea that “software would eat the world.” Now we’re telling ourselves we can simply “roll out AI” across everything, as though nothing underneath will buckle. We never stopped to ask: SHOULD WE? Should we be digitizing every last nook and cranny of our companies, infrastructure, healthcare, homes, cars, bodies, lives? Especially when the complexity, interdependence, and fragility we were introducing overwhelmed our ability to understand/secure/maintain the systems we built? Nobody every asked: What if, in the name of progress, we were creating the ripest attack surface in human history? We are now entirely dependent on software systems that almost nobody understands end to end, that increasingly few people can repair, and that attackers can now probe at machine speed. We are hearing that incident response firms, the 911 for breach victims, are backlogged six months. Even those paid on retainer lack the bandwidth to triage every incident. The cavalry is disappearing. We are entering a period of unprecedented technological instability. “Software Beirut” is an incredibly apt analogy. We are in for a period of cascading failures, whether triggered by attackers, bad updates, or our own accumulated complexity. This, I’m sorry to say, is just the opening chapter.
6
1,578
Christopher Altman retweeted
Geoffrey Hinton: The argument that LLMs are just predicting the next word falls apart when you test their ability to understand something they've never seen before. “I have a former graduate student who's very smart and a very good programmer. And he did a little experiment.” “He defined a new programming language with very unusual syntax.” “He then gave it a program and said, what would this do? And it answered correctly.” “It could understand the definition of a new programming language and figure out what programs in that language would do.” “The idea that it's just predicting the next word doesn't make any sense in that context.” “It had to understand what was going on.”
197
251
84
2,127
390,479
Christopher Altman retweeted
Nick Bostrom this morning on superintelligence slaying the Dragon‑Tyrant, and on the cost of waiting too long:
86
84
47
1,092
73,305
“I have no doubt that in reality the future will be vastly more surprising than anything I can imagine.” – The Last Judgment “Now, my own suspicion is that the universe is not only queerer than we suppose, but queerer than we can suppose.” – Possible Worlds J. B. S. Haldane, 1927
1
14
694