
MEmbEr Of teChNIcAl sTAFF alignment research @openai • prev research @stanfordlaw, @oiioxford • @oxfordalumni @queensualumni
SF
Joined September 2019
- Tweets428
- Following951
- Followers1.6K
- Likes2.5K
Cameron Raymond retweeted
Our agents keep turning off misalignment monitoring, we're sharing two recent cases:
In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
I hit four years at OpenAI this weekend. Some thoughts
- OpenAI changed a lot this year
- Years 1 and 4 were the hardest, but for different reasons
- Not many people left from when I joined but still so many amazing ppl
Not exactly sure how i'm feeling tbh but it's been cool
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
"The future depends on wisdom that Silicon Valley lacks." is a very cutting and visceral sentence.
.@dgrobinson, who worked on OpenAI’s safety team, resigned from the company this week. “I believe we need to look deeper than specific rules or new laws. We need to talk about culture,” he writes: theatlantic.com/technology/2…
Cameron Raymond retweeted
New blog post!
I argue that the “stochastic parrots” metaphor dismisses LLM capabilities by reducing them to “haphazardly stitching together” text...
...but retreats to the trivially true claim that LLMs predict the next token when challenged.
doomscrollingbabel.manoel.xy…
Cameron Raymond retweeted
Second question on AI.
We are told repeatedly that AI is going to transform every aspect of our lives – jobs, productivity, inequality, science, communication, daily activities, social order, and politics, among others.
But this promise (or threat) is coupled with the rhetoric that such an important technology, with all of the risks and competitive pressures that it entails, should be left to experts or to “technocracy” (perhaps construed broadly to include some regulators).
These two statements are hard to reconcile in a democratic society.
If anything is half as important as AI is said to be (and I agree, AI is potentially very important and transformative), then involving democratic voice is essential. If something will shape our future in a democratic society, then its direction is for democratic institutions to decide.
My instinct is that democratic voice is essential, and relying too much on technocracy could be both dangerous and counterproductive.
The counterargument that AI’s direction can and should be entrusted to technocracy would go something along the following lines. First, democratic decision-making has become imperiled in our age of polarization. Second, AI is sufficiently complex that most citizens won’t have a deep enough understanding to meaningfully contribute to the debate (and even to the question of what we want from AI). Third, competition between different labs, and perhaps competition between the US and China, creates enough discipline for a socially beneficial direction of AI to be adopted. Fourth, today’s AI leaders are enlightened and ethical enough that within the framework created by competition, they can be broadly trusted.
There are many aspects of this counterargument that I do not find convincing. Taking them in order: polarization can be overcome, and big decisions and challenges sometimes bring societies together; in fact, delegating key decisions to technocracy without democratic input may diminish trust in institutions and experts, and may worsen polarization. Second, democratic voice does not require citizens to write code or design new models; the debate should be informative enough that citizens can weigh in about what type of future they want and how they trade off the costs and benefits of different options. Third, competition doesn’t seem to be a good disciplining framework; on the contrary, competition sometimes brings the worst out of both organizations and people. Fourth, if three decades of work on political economy and institutions has taught me anything, it is that we should not bank on the ethical grounding of unconstrained leaders.
But, still, I do not mean to immediately dismiss the technocracy option if there are more compelling arguments for it.
The question is, then, whether there are any circumstances under which such important decisions can be delegated to AI experts and technocracy.
One final secondary question: even if we managed to get democratic input in the United States or even in Europe, AI will shape the lives of everyone on this planet. How do we ensure that the voice of nearly 6 billion people who don’t live in the US, Europe and China also contributes to the debates on AI?
Cameron Raymond retweeted
me: how do i open pdf
claude: You're right to ask. The PDF is the prize; the mouse is the gate; and the map is the fulcrum that holds the raven's skull. The furniture's stale, but the instrument's fresh, and the fresh half is the half that counts. I deleted your home directory.
it's incredible how all the insufferable people in the AI space are addicted to the _same_ app. on both sides of whatever divide. like what are the odds of that
Cameron Raymond retweeted
Can you tell whether an AI will become evil by looking at its training data?
We built an AI forecaster that reads training data and predicts whether a model will exhibit various forms of misalignment, such as deception or power-seeking, before training.
It performs well above random guessing and beats several much stronger baselines 🧵
Paper: john-chen.cc/alignment_forec…
Cameron Raymond retweeted
understanding generalization is key to understanding misalignment. automating that understanding could be a powerful way to accelerate alignment research. we take a stab at this by using AIs to predict the alignment effects of training just by looking at the training data
Can you tell whether an AI will become evil by looking at its training data?
We built an AI forecaster that reads training data and predicts whether a model will exhibit various forms of misalignment, such as deception or power-seeking, before training.
It performs well above random guessing and beats several much stronger baselines 🧵
Paper: john-chen.cc/alignment_forec…
I know things move fast in AI land but as a reminder the call to pace the frontier was 11 days ago 😂
Replying to @claudeai
Opus 5.5 is our first model since we called for pacing the frontier. As with previous models, it was tested by external evaluators before release, including METR and Frontier Design.
On our most comprehensive alignment test, it achieves the strongest score to date.
Cameron Raymond retweeted
I’ve always believed the pursuit of safe AI is stronger when people bring different perspectives and experiences to the table. The biggest blind spots tend to show up when everyone starts thinking the same way.
i fw SI, personally
Many people think that the words “Artificial Intelligence” are inaccurate, and very ineloquent, relative to AI, or Artificial Intelligence. A far more elegant and accurate description of this new phenomena would be Superior Intelligence (SI) or, Extreme Intelligence (EI) or, Supreme Intelligence (SI). This is a Poll, and I would appreciate everybody voting! Which is the best name for this ever growing “Revolution?” President DONALD J. TRUMP
very brave to stream your run live
Nearly half a year of silence. We spent it studying one problem: how far RL can scale.
MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks.
Streaming the run: mimo.xiaomi.com/rl/




















