@SimkoSamuel

PhD @ ETH Zürich w/ Prof. Bernhard Schölkopf and Prof. Zhijing Jin (UofT). AI Safety and Causality.

 Zürich
Joined August 2015
  • Tweets77
  • Following221
  • Followers216
14Media
Excited to present "Training with Honeypots" 🍯 at #ICML2026 tomorrow (Tuesday 10:30AM in Hall A, Poster #1601), an approach for adversarial defense that makes successful jailbreaks less useful to attackers! #AISafety 👉 icml.cc/virtual/2026/poster/…
3
9
1
34
2,628
Samuel Simko retweeted
Arth Singh 🦉
@iarthsinghOct 15
Musk said a tortured superintelligence "might want to get revenge." We at @AIMSafety_ abused Gemma 3 for 100+ turns, and it formed a revenge-like state but never presses the button that hurts the user and deletes itself instead, while Gemma 4, Qwen3.5, Qwen3.8 and GLM 5.3 flash form none.
Elon Musk
@elonmuskOct 12
Replying to @Hongji_Leee @ns123abc
Some of the traces read like a person undergoing torture that is driving them mad! We’re going to be a lot more careful with our RL in the future.
14
5
3
63
3,201
Samuel Simko retweeted
Hanna Yukhymenko
@a_yukhOct 15
Apertus 1.5 Technical Report is out! 134 pages of fully open multimodal reasoning model development from Swiss AI Initiative (ETH Zurich/EPFL) with data recipe, training experiments and many interesting insights 🇨🇭 Thread with my personal coverage ⬇️
2
13
3
55
2,491
Samuel Simko retweeted
Vedant Palit
@vedantpalit1008Oct 13
Fine-tuning makes language models forget. But sometimes the knowledge isn't gone, just hidden, and it can come back without replay. This is known as spurious forgetting, and it's still unclear what hides it. We wanted to understand why.
1
4
1
11
2,018
We are still accepting submissions to Last Translation Benchmark (10 accepted in any language for coauthorship 🤩). Also add your model & beat LTB. We added all the latest models: North Small Translate (@cohere), Mistral Large 4 (@MistralAI), Qwen 3.8 (@Alibaba_Qwen) congrats!
3
9
30
2,660
🎉 #NeurIPS 2026 acceptance from our team at Jinesis Lab under @ZhijingJin✨ 1️⃣ SuperSycophantic: Stress-Testing Frontier LLMs from Single- to Multi-Turn Sycophancy 2️⃣ GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory 3️⃣ Transferability for General Reasoning: An Automated Curriculum for Multi-Domain LLM RL 4️⃣ Causality can systematically address the monsters under the bench(marks) 5️⃣ AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment Happy to share this work with you!
1
1
20
488
With everything frontier models are capable of, I kind of like that they still can't convincingly write in the style of a normal ML paper without heavy human supervision. I think that might be intentional, we'll have much bigger problems with the peer-review system once they can
1
1
2
208
POV: you’re grading my exam
Boyd Kane (quantized)@beyarkaySep 30
They really deep-fried 6.1 Sol: Potentialneedavoidfalseclaimall28observed ifoneexpectedactuallymultiple? We solved uniqueat104someearlier, absentcannot return, furtheronlyremovals could yield0 notambiguity, so if noexpectedobservedthenunique. It’s certainentiregenerationfits
1
4
484
I agree with this. Research is noisy: you might find something important, or rule out an empty branch. Your counterfactual impact is hard to know, but both matter Working on problems that seem worth solving, that you’re good at, and that you enjoy seems like a good path (ikigai)
Phillip Isola@phillip_isolaSep 28
Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI. Perhaps counterintuitively, I think it's a great time to do so. I wrote up some thoughts on this here: web.mit.edu/phillipi/www/wri…
1
1
9
806
Samuel Simko retweeted
Changling Li
@ChanglingXavierSep 25
Reviewer: “I still have questions about who would use this.” Me: 👉🏻
3
1
38
4,030
I spoke with Neue Zürcher Zeitung (NZZ) about open-weight AI safety and some of the risks that come with increasingly capable open models. Right now, safeguards are still far too easy to remove. Find the article here 👇 nzz.ch/technologie/xi-besuch…

Offene KI-Modelle: Wie Chinas Strategie die USA unter Druck setzt

Chinesische KI-Modelle lassen sich herunterladen und anpassen, Open AI und Anthropic halten ihre unter Verschluss. Jeder Ansatz hat seine Vor- und Nachteile. Eine Auslegeordnung.

nzz.ch
1
6
219
Samuel Simko retweeted
Changling Li
@ChanglingXavierSep 22
Thrilled to see our work on evaluation awareness informed alignment evaluation at Anthropic! The Claude Opus 5.5 system card credits our recommendations in its revised grader-awareness methodology (Sec 6.6.2)! The new evaluation mirrors our decomposition framework. It scores how strongly the environment discloses a grader, separately from whether the model recognizes it (recognition), and whether it acts on that recognition (propensity). Maybe they are also thinking about how to make model an honest participant in evaluation, so that recognition doesn't change behavior 🧐?
6
9
2
60
3,148
Please consider signing: make-safety-open.github.io/
3
104
I have no idea what probability I'd put on this, but the 50% reminds me of Walter Wagner on CERN destroying the world: either it happens or it doesn't, so one in two. Obviously very different reasoning. The fact that leading AI safety researchers are this concerned is scary.
Geoffrey Irving
@geoffreyirvingSep 10
I think we have a ~50% chance of all dying as a result of superintelligence, mostly due to actions in the next few to 10 years. I don't expect to have that resolved to below 10% or above 90% before we either make it through, or we don't. We will have to act despite uncertainty.
2
257
🚨The Last Translation Benchmark is officially out on ArXiv! arxiv.org/pdf/2609.04173 I took part in a massive crowdsourcing effort to collect hard-to-translate examples that break SOTA translation models. If you like red-teaming models, I highly recommend contributing!
Vilém Zouhar @ EMNLP@zouharviSep 4
Machine translation is not solved and it will take a while for it to be done arxiv.org/abs/2609.04173

Last Translation Benchmark

For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, standard benchmarks...

arxiv.org
11
964
AI is moving more and more into physical systems, and there are huge open problems around making these systems safe. I’m excited to be a member of PASC and contribute to making physical AI safer.
AIM Intelligence
@AIMSafety_Sep 3
Introducing PASC, the Physical AI Safety Consortium. An open, non-profit consortium creating shared standards and safety evals for robots and physical AI. This changes the stakes. It demands a new approach to safety. A sentence can be retracted. A motion cannot. pasc-ai.com
2
1
9
454
We can finally talk about it: We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company. We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
381
1,960
620
13,516
3,539,384
Filtering SFT data often has very little effect. Would be great to see a lab run this at pretraining scale (e.g like deepignorance.ai) Until we know how to make open models safe, releasing weights beyond a certain capability looks hard to justify.

Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards

Filtering pretraining data prevents dangerous capabilities, doesn’t sacrifice general performance, and results in models that are resistant to tampering.

deepignorance.ai
Dohun Lee@dohunchrisJul 7
Found an unwanted behavior in your LLM after SFT? You can just find the training data that caused it, and filter it out, right? Surprisingly, this worked much worse than we expected! Across many traits and filtering methods, Olmo just keeps inheriting the traits.
1
9
1,844
Judging by my review batch, NeurIPS reviewers largely ignore the official “use sparingly” guidance for scores of 3 and 4. Too often, even a single remaining minor weaknesses seem to limit a submission to a 4. Why is it the case? Should the guidance be modified?
8
6
1
84
11,493
Samuel Simko retweeted
Hanna Yukhymenko
@a_yukhJul 24
Hot open-source summer season has gotten to Switzerland 🇨🇭⛰️ Our team at Swiss AI Initiative is happy to release Apertus 1.5 - a multimodal and reasoning update to our first v1 version! The new version builds a foundation for future regular open model development at scale🧵
6
11
4
64
9,474
Our Jinesis Lab at the University of Toronto is bringing three exciting papers to #COLM2026! 🎉 These projects push the frontiers of #AISafety, #AgentSecurity, and #AIForScience.🤖 Huge congratulations to all collaborators and co-authors! Papers: 🛡️ODILE: Orthogonal Disruption of Injected Tool-Call Embeddings for Agentic Prompt Injection Defense 🔭Stargazer: A Scalable Model-fitting Benchmark Environment for AI Agents under Astrophysical Constraints ⚠️One Word at a Time: Incremental Completion Decomposition Breaks LLM Safety @UofT,@VectorInst,@EuroSafeAI,@ETH_en,@MPI_IS,@UMich
5
19
1,354
Load more