A non-profit AI Alignment organization based out of the UK.
Cambridge, UK
Joined August 2025
- Tweets107
- Following179
- Followers934
- Likes222
We excited to announce two fellowship streams this winter with @pivotal_org. Come help us with AI Alignment research during some of the most important months in human history.
Applications for this round close this Sunday. Link to apply below!
The other, led by Kyle O'Brien, you'll be studying the question: “Since we are concerned about X, is pretraining on X-relevant data making things worse?"
Successful projects here inform best practices in both Geodesic and frontier pretraining runs.
pivotal-research.org/mentors…
Both streams will be supported by @RadmardPuria and will tie into the main lines of inquiry being pursued by the Geodesic team at large.
Filtering subversion related information from pretraining is feasible.
We're releasing a research note where we remove information related to monitoring designs and successful subversion strategies in three 30B LLMs.
In collaboration with @redwood_ai 🧵
Authors: Kyle O'Brien, @SpencerKitts, @cam_tice, and @AlekWestover
Thanks for feedback from: @jzlucassen, @ejcgan, @LukasFinnveden, @RyanGreenblatt, @bshlgrs, @tomekkorbak, @edwardjyoung, @RadmardPuria, @DavidDAfrica , and @jammastergirish
See the full research note here: substack.com/home/post/p-219…
And LW blog here: lesswrong.com/posts/HwdXDoCH…
There's a 𝐠𝐫𝐨𝐰𝐢𝐧𝐠 𝐫𝐞𝐬𝐨𝐮𝐫𝐜𝐞 𝐠𝐚𝐩 𝐛𝐞𝐭𝐰𝐞𝐞𝐧 𝐬𝐚𝐟𝐞𝐭𝐲 𝐭𝐞𝐚𝐦𝐬 𝐢𝐧𝐬𝐢𝐝𝐞 𝐟𝐫𝐨𝐧𝐭𝐢𝐞𝐫 𝐥𝐚𝐛𝐬 𝐚𝐧𝐝 𝐢𝐧𝐝𝐞𝐩𝐞𝐧𝐝𝐞𝐧𝐭 𝐬𝐚𝐟𝐞𝐭𝐲 𝐫𝐞𝐬𝐞𝐚𝐫𝐜𝐡𝐞𝐫𝐬, and closing it is essential to maintaining an ecosystem of impactful safety research.
We've spent months securing a large, multi-year GPU contract. This taught us that 𝐟𝐮𝐧𝐝𝐢𝐧𝐠 𝐢𝐬 𝐧𝐞𝐜𝐞𝐬𝐬𝐚𝐫𝐲 𝐛𝐮𝐭 𝐧𝐨𝐭 𝐬𝐮𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭. We've written up the non-obvious bottlenecks we hit, and what can be done to alleviate them: lesswrong.com/posts/aCGx79eG…
Compute is only half of it. Labs' internal coding agents are likely well ahead of publicly available tools, and safeguards throttle safety work that looks like frontier development; we've hit this on numerous occasions optimising infrastructure. You can't buy your way to parity with token spend.
Our appendix includes the full story of the deal, our procurement strategy and takeaways. Read the full: lesswrong.com/posts/aCGx79eG…
