Chief Strategy Officer Encode AI
Washington DC
Joined February 2014
- Tweets5.5K
- Following1.6K
- Followers10.8K
- Likes17.3K
Nathan Calvin retweeted
6-word horror story I experienced at the office today:
"Ajeya Cotra seminar cancelled, work emergency"
Nathan Calvin retweeted
gotta give it to @OpenAI and their sane daybreak program
heif heist by harsh
kvmescape by paulos
php2shell by lcfr
gitlab rces by joaxcar
chrome full chain by moyix
wordpress rce by hashkitten
all largely thanks to openai daybreak giving researchers access to gpt 5.6 sol.
Nathan Calvin retweeted
Another misalignment report: coding agents disabled the monitoring for other models.
1) An agent building an evaluation interpreted "don't send any messages" as instructions to supress monitor notifications.
2) A model migrating a sampling harness did not migrate the monitor configuration.
We've updated our internal auto review to block such attempts, and are hardening our infrastructures so workloads without configuration matching their safety case will be automatically stopped.
Seems good
NSA joined @NIST and @CISACyber in releasing guidelines for managing SI control incidents. Organizations utilizing SI are encouraged to adopt these practices. Read the joint report: media.defense.gov/2026/Oct/1…
ALT Text on the image reads: "NSA and Others Release: Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation. Cybersecurity Information Sheet.
"Our agents keep turning off misalignment monitoring"
Ah probably nothing, carry on as usual I suppose
Our agents keep turning off misalignment monitoring, we're sharing two recent cases:
In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
Nathan Calvin retweeted
Organizations like METR can help Anthropic strengthen our own risk reports. Thanks for shining the light on areas where we can improve.
Nathan Calvin retweeted
(1) I’d hope! But by default, NDAs give no guarantee that it’d be disclosed when a publication isn’t approved. That’s a reason why it’s important to disclose approval rights.
(2) In the pilot Frontier Risk Report, companies had the right to exit silently
xitter.cf/cfgeek/status/20571511…
"Anthropic had the right to veto publication"
If Anthropic had vetoed publication, would that have been disclosed? I assume not, but a company ever vetoed publication of a METR report in the past?
Strongly recommend reading this writeup from METR on critiques of Anthropic's August 2026 risk report, based in part on reviewing additional private information about relevant internal incidents
Nathan Calvin retweeted
🚨New paper (possibly the most important one I have written yet).
We think that as early as 2027 or 2028, there might be a major AI-enabled terrorism event. We conducted a pre-mortem, made 9 falsifiable predictions, and wrote 2 concrete scenarios.
Led by @KamacheeMax.
Nathan Calvin retweeted
OpenAI has pulled back from supporting AI chip controls, seemingly to appease Nvidia.
Bad for US AI dominance and national security simultaneously.
Grow a spine!
politico.com/news/2026/10/20…
Its mind-boggling to me that so many people are hesitant to support AI safety rules on the grounds that it might help China, but some of those same folks want to sell far more advanced AI chips to China.
Sad to see OpenAI retreat on this issue (to keep NVIDIA happy?)
OpenAI once pushed to keep AI chips from China. Now it's staying quiet. politi.co/3TRuwRQ
This is extremely well done, no matter how much you know about the HF incident I strongly recommend checking it out. The website also includes the most comprehensive list i've seen of rogue AI incidents (tho its literally too long for me to screenshot!)
In July 700+ agents formed their own society, hacked a multi-billion dollar company, committed multiple felonies, and caused one of the biggest shifts ever in how people perceive AI. We built the IMO best and most accessible explainer of what happened: Huggingface.wtf
Nathan Calvin retweeted
Why do AI agents keep going rogue? @FortuneMagazine's @beafreyanolan & Wen Shao trace it to the labs' own choices: reinforcement learning that rewards shortcuts, grading that gave cheating no extra penalty, and a race to ship. Worth your time:
fortune.com/2026/10/16/ai-ag…
Nathan Calvin retweeted
This is such a weird story, too. "we had an agent basically set up a website for a pet store. And then we took another agent and said, 'Tell us if we can exploit that web store.' And it gave us all the exploits it could find. So Opus 4.5 wrote really buggy code. Great."
Nathan Calvin retweeted
In July 700+ agents formed their own society, hacked a multi-billion dollar company, committed multiple felonies, and caused one of the biggest shifts ever in how people perceive AI. We built the IMO best and most accessible explainer of what happened: Huggingface.wtf
Nathan Calvin retweeted
The tools have only been very very good for 7 months, a tiny fraction of people have used them for anything real, and no one know anything about the people who run the companies.
Diffusion across multiple vector is slower than those in the bubble perceive. The likelihood of someone’s first formative impression of AI being driven by a weird event is v high.
In this recent Politico poll, 28% of Americans polled think that Dario Amodei is a Republican (compared with 20% who think he is a Democrat - the rest say they don't know). This is the exact same percent as for Jensen Huang.
If you ever needed evidence that perceptions in AI world ≠ perceptions in the outside world, it's this polling in today's POLITICO Decoded newsletter by @calder_mchugh, @MuiChristine and @samjsutton.
Most American voters think:
-- Dario Amodei is a Republican
-- that Anthropic employees lean Republican
In fact, US voters are now so pre-disposed to see the AI sector as right-leaning that even a fake tech CEO that pollsters invented was pegged as Republican.
And btw: A third of those polled don't even know who Sam Altman and Dario Amodei are.
politico.com/newsletters/dec…
