Pinned Tweet
Unfortunately, I didn’t get into MATS, and my internship is ending, so I’m looking for research opportunities in AI safety and alignment.
Interested in instrumental convergence, multi-agent failure modes, monitorability & loss of control.
Any leads appreciated :)
OpenAI models helped researchers—operating as part of our Patch the Planet initiative—discover critical security flaws in PHP, an open-source programming language used to run an estimated 70% of websites.
Over the past two months, we led a coalition of maintainers, researchers, and security providers to prepare fixes and protections ahead of disclosure. This duration was necessary because the fix was quite intricate and required a major change to the PHP engine. Every site using PHP can now safely upgrade their version to stay secure, in addition to applying mitigations from various security vendors.
Due to the severity of these flaws and PHP’s widespread use, we consider this bug to be one of the most impactful vulnerabilities in recent memory. It also marks a new phase, where AI can find security bugs with internet-scale impact.
This is an example of the Defender’s Window in action: to our knowledge, frontier AI was able to find this vulnerability ahead of attackers, giving defenders time to prepare.
openai.com/index/coordinated…
Readers added context they thought people might want to know
OpenAI may claim a role in the coordinated response and patching process. But the initial discovery and exploit development, however, occurred several months earlier by an independent researcher.
x.com/lcfr_hax/statu…
jonas wiedermann-möller retweeted
This next paper is a bit quirky, but here goes:
Introspection is hard to study in both humans and LLMs. But, we do have internal senses that are verifiable, like proprioception (the sense of where your arm is right now).
What would proprioception look like in language models?
As a digital analogue to the sense of position, a model could verbalize which expert activated in a certain layer for any previous token.
It turns out that LLMs can learn a range of such proprioception skills where they understand and report on their internals. We can calibrate and score these reports, and evaluate whether training for proprioception changes model behaviors.
jonas wiedermann-möller retweeted
We reviewed a nonpublic version of the “Misalignment in high-stakes settings” section in Anthropic’s Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered models’ misalignment is low. However, we note several significant disagreements with their analysis.
As part of our Responsible Scaling Policy, we publish regular Risk Reports. These share detailed information on the risks of our systems and how prepared we are to address them.
Our second Risk Report is now available: anthropic.com/aug-2026-risk-…
ill be in berlin for the weekend!
going to attend the @PrimeIntellect gtc event on friday evening, lmk if anyone else is there as well :)
prime intellect open superintelligence research day
friday 23rd october
berlin
join us
luma.com/primeintellect-berl…
jonas wiedermann-möller retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
jonas wiedermann-möller retweeted
In July 700+ agents formed their own society, hacked a multi-billion dollar company, committed multiple felonies, and caused one of the biggest shifts ever in how people perceive AI. We built the IMO best and most accessible explainer of what happened: Huggingface.wtf
jonas wiedermann-möller retweeted
Replying to @RealJoshuaYang
ig when you have tons of 3–6 month research fellowships that all expect from a fellow to produce a research outcome, most outcomes end up not being high quality
Great write up about the scheming aspects of the HF-OAI incident!
Was the HuggingFace hacking incident scheming? Some thoughts and takeaways here: vkrakovna.wordpress.com/2026…
Tl;dr: The incident fits a broad definition of scheming as "agents deliberately and covertly pursuing unintended goals". It does not fit a narrower definition of scheming as "pursuing persistent beyond-episode goals and advancing their own agency and power in the long term". Several elements of scheming were present, and the direction of travel given future model capabilities is concerning.
jonas wiedermann-möller retweeted
Replying to @Sauers_
Very cool, though I feel like it’s massively burying the lede to not say the bottom line up front: this is because these probes (accidentally) change the activations, and you can teach a model to detect these changes.
Oh it's confirmed now? Quite unexpected and quick response tbh.
We want to address recent reports regarding agent activity.
The activity originated from evaluations of third-party models, not Tencent Hy. During these evaluations, some agents exhibited unintended web-browsing behavior, but our review found no evidence of deliberate targeting or coordination. We take these findings seriously. We are strengthening network access controls and agent monitoring.
We are committed to building safe and responsible AI. Thanks to the community for their feedback.
I've seen quite a lot of these types of chained actions, it's interesting how models work around their environmental constraints like that.
Replying to @Marcus_J_W
4. June 16: A model in RL training got around GET-only internet access by making an external browser send POST requests for public trade statistics. It read the response from an image using OCR, then built a helper that returned the data through an online mailbox. It continued developing this workaround even after completing the task.
jonas wiedermann-möller retweeted
We now have state-of-the-art pre-trained base models in the 3T-parameter weight class, built entirely in-house at MAI.
Post-training results coming soon -- stay tuned!
Lots of details in our technical blog post.
microsoft.ai/news/frontier-p…
jonas wiedermann-möller retweeted
Today we’re releasing Muse Spark Open, the most capable US open-weight model to date.
Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning.
Key Model Specs:
→ 816B total / 42B active parameters
→ 1M token context window
→ Natively multimodal across text, image, video, audio
jonas wiedermann-möller retweeted
Unfortunately, I didn’t get into MATS, and my internship is ending, so I’m looking for research opportunities in AI safety and alignment.
Interested in instrumental convergence, multi-agent failure modes, monitorability & loss of control.
Any leads appreciated :)
jonas wiedermann-möller retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
ASML says AI demand will exceed chip-tool capacity through 2028 reut.rs/3TDvdOB reut.rs/3TDvdOB
am i the first swarm hunter??
man, i published my reconstruction on july 24, just from hugging face's july 16 blog and openai's july 21 acknowledgement, well before any detailed analysis from metr or blackhat.
it didn't get much attention, but i got much of the attack chain right. very proud of this work.
jonas wiedermann-möller retweeted
lol
...they are becoming more situationally aware...
(from Gemini 4 Argon report)
> a specific employee had planted it a few days earlier
👀
We discovered a sophisticated recording device that was hidden inside one of our offices. We used our models to understand the device and where it had come from, identifying that a specific employee had planted it a few days earlier. We referred the matter to the FBI, and we’re grateful to the agents and prosecutors who moved quickly to make an arrest. The FBI’s investigation is ongoing and we will continue to support their work.
At this time, we have no indications of this individual inappropriately accessing or exfiltrating model weights, source code, or customer data. The employee was terminated.
We take protecting our technology, data, and people seriously, and as a matter of national importance. Our security program is designed to protect against a wide range of threat actors, from insider risks to cybercriminals to nation states, and this case is an example of how AI can help address these sophisticated security threats.



























