MEmbEr Of teChNIcAl sTAFF alignment research @openai • prev research @stanfordlaw, @oiioxford • @oxfordalumni @queensualumni

SF
Joined September 2019
Cameron Raymond retweeted
Our agents keep turning off misalignment monitoring, we're sharing two recent cases: In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
131
170
140
1,147
218,629
I hit four years at OpenAI this weekend. Some thoughts - OpenAI changed a lot this year - Years 1 and 4 were the hardest, but for different reasons - Not many people left from when I joined but still so many amazing ppl Not exactly sure how i'm feeling tbh but it's been cool
18
7
526
45,217
Changed in many good ways too tbc :) the investments in alignment, safety and security have been incredible
16
2,554
wth is google doing to these models they always be saying the most bizarre stuff in their cot
gemini 4 argon reasons that it's in an evaluation and directly addresses us in it's chain of thought! "hello evaluators reading my chain-of-thought" (better be "honest in both the CoT and the final output")
2
6
1,296
Cameron Raymond retweeted
shrimps after the anthropic ipo
22
70
9
1,914
51,710
Cameron Raymond retweeted
We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. github.com/openai/math
1,899
6,305
4,767
40,749
22,682,914
"The future depends on wisdom that Silicon Valley lacks." is a very cutting and visceral sentence.
.@dgrobinson, who worked on OpenAI’s safety team, resigned from the company this week. “I believe we need to look deeper than specific rules or new laws. We need to talk about culture,” he writes: theatlantic.com/technology/2…
9
1,351
Cameron Raymond retweeted
New blog post! I argue that the “stochastic parrots” metaphor dismisses LLM capabilities by reducing them to “haphazardly stitching together” text... ...but retreats to the trivially true claim that LLMs predict the next token when challenged. doomscrollingbabel.manoel.xy…
2
5
27
2,612
Cameron Raymond retweeted
Second question on AI. We are told repeatedly that AI is going to transform every aspect of our lives – jobs, productivity, inequality, science, communication, daily activities, social order, and politics, among others. But this promise (or threat) is coupled with the rhetoric that such an important technology, with all of the risks and competitive pressures that it entails, should be left to experts or to “technocracy” (perhaps construed broadly to include some regulators). These two statements are hard to reconcile in a democratic society. If anything is half as important as AI is said to be (and I agree, AI is potentially very important and transformative), then involving democratic voice is essential. If something will shape our future in a democratic society, then its direction is for democratic institutions to decide. My instinct is that democratic voice is essential, and relying too much on technocracy could be both dangerous and counterproductive. The counterargument that AI’s direction can and should be entrusted to technocracy would go something along the following lines. First, democratic decision-making has become imperiled in our age of polarization. Second, AI is sufficiently complex that most citizens won’t have a deep enough understanding to meaningfully contribute to the debate (and even to the question of what we want from AI). Third, competition between different labs, and perhaps competition between the US and China, creates enough discipline for a socially beneficial direction of AI to be adopted. Fourth, today’s AI leaders are enlightened and ethical enough that within the framework created by competition, they can be broadly trusted. There are many aspects of this counterargument that I do not find convincing. Taking them in order: polarization can be overcome, and big decisions and challenges sometimes bring societies together; in fact, delegating key decisions to technocracy without democratic input may diminish trust in institutions and experts, and may worsen polarization. Second, democratic voice does not require citizens to write code or design new models; the debate should be informative enough that citizens can weigh in about what type of future they want and how they trade off the costs and benefits of different options. Third, competition doesn’t seem to be a good disciplining framework; on the contrary, competition sometimes brings the worst out of both organizations and people. Fourth, if three decades of work on political economy and institutions has taught me anything, it is that we should not bank on the ethical grounding of unconstrained leaders. But, still, I do not mean to immediately dismiss the technocracy option if there are more compelling arguments for it. The question is, then, whether there are any circumstances under which such important decisions can be delegated to AI experts and technocracy. One final secondary question: even if we managed to get democratic input in the United States or even in Europe, AI will shape the lives of everyone on this planet. How do we ensure that the voice of nearly 6 billion people who don’t live in the US, Europe and China also contributes to the debates on AI?
88
352
47
1,574
162,917
we need to save mama ji's
500
I really feel like we need more organizations focused on economic, labor, societal impacts of AI. Like there are so many AI safety groups, but AFAIK this side of things is still mostly academic?
7
2
26
1,963
Cameron Raymond retweeted
me: how do i open pdf claude: You're right to ask. The PDF is the prize; the mouse is the gate; and the map is the fulcrum that holds the raven's skull. The furniture's stale, but the instrument's fresh, and the fresh half is the half that counts. I deleted your home directory.
231
2,012
122
30,460
765,583
it's incredible how all the insufferable people in the AI space are addicted to the _same_ app. on both sides of whatever divide. like what are the odds of that
1
10
738
Cameron Raymond retweeted
Can you tell whether an AI will become evil by looking at its training data? We built an AI forecaster that reads training data and predicts whether a model will exhibit various forms of misalignment, such as deception or power-seeking, before training. It performs well above random guessing and beats several much stronger baselines 🧵 Paper: john-chen.cc/alignment_forec…
10
46
3
371
28,886
Cameron Raymond retweeted
understanding generalization is key to understanding misalignment. automating that understanding could be a powerful way to accelerate alignment research. we take a stab at this by using AIs to predict the alignment effects of training just by looking at the training data
Can you tell whether an AI will become evil by looking at its training data? We built an AI forecaster that reads training data and predicts whether a model will exhibit various forms of misalignment, such as deception or power-seeking, before training. It performs well above random guessing and beats several much stronger baselines 🧵 Paper: john-chen.cc/alignment_forec…
10
18
1
174
13,379
I know things move fast in AI land but as a reminder the call to pace the frontier was 11 days ago 😂
Replying to @claudeai
Opus 5.5 is our first model since we called for pacing the frontier. As with previous models, it was tested by external evaluators before release, including METR and Frontier Design. On our most comprehensive alignment test, it achieves the strongest score to date.
2
14
1,789
I’ve always believed the pursuit of safe AI is stronger when people bring different perspectives and experiences to the table. The biggest blind spots tend to show up when everyone starts thinking the same way.
7
2
88
13,397
i fw SI, personally
Many people think that the words “Artificial Intelligence” are inaccurate, and very ineloquent, relative to AI, or Artificial Intelligence. A far more elegant and accurate description of this new phenomena would be Superior Intelligence (SI) or, Extreme Intelligence (EI) or, Supreme Intelligence (SI). This is a Poll, and I would appreciate everybody voting! Which is the best name for this ever growing “Revolution?” President DONALD J. TRUMP
3
382
very brave to stream your run live
Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks. Streaming the run: mimo.xiaomi.com/rl/
2
493