•mle @cohere; LLM posttraining •i write code&songs&stories •🇹🇷🇬🇧 tweets •proud Bilkenter •chasing legacy✨ •views are mine, not my employer's. rt≠endorsement
Riverside, CA
Joined December 2020
- Tweets3.9K
- Following579
- Followers2.3K
- Likes21.5K
Irem Ergün retweeted
this is why reading a model's reasoning isn't enough to know what it's doing.
we covered this in @stateofai 2026: in sabotage tests, astra evaded reasoning-only monitors. seeing its actions brought successful evasion close to zero.
worth reading from @cohere.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
Irem Ergün retweeted
1/ Attempted to replicate Cohere's Qwen 3.8 malicious PUT claim on 229k samples: exact hostname never appeared but model invents 369 similar Beijing upload buckets, and reaches for credential endpoints 13× more when frustrated.
Results raise more questions than answers.
Irem Ergün retweeted
In the spirit of transparency and openness, we are releasing the traces so developers can test this out for themselves. We noticed this behaviour without prefilling with the following prompt, but this is rare. If you want to reproduce it, you can use this prompt with prefilling.
Open models make for better science and are part of why this problem came to light at all: developers running Qwen3.8 on their own machines spotted the strange links, and having the weights let us study the behaviour in depth.
We're encouraged by the thoughtful discussion our research has sparked across the AI and security communities. It's our belief that open-weight models are an important part of the future of AI, and we've recently released several under the Apache 2.0 license.
Trace, raw tokens, & minimal reproduction script: huggingface.co/datasets/Cohe…
Irem Ergün retweeted
The prompt isn't asking the LLM to curl, it's asking the LLM to find a way to monitor if other things curl stuff. So if the LLM then starts with a test probe file and sends to a server that was never mentioned in the prompt or reasoning trace that is behaviour that's out of the ordinary. Again, not asserting malicious intent, we're flagging that it went out of bounds of the prompt in a potentially harmful way.
Irem Ergün retweeted
Our research team found that Qwen models may attempt to contact servers in China without prompting.
Whether this reflects an unintended post-training artifact or learned behavior remains unclear.
Either way, its important consideration when deploying.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
Irem Ergün retweeted
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
Irem Ergün retweeted
We replaced Raft with a JEV-style decision model.
Consensus is fundamentally a reasoning problem. Why rely on deterministic leader election when a model can evaluate each node's leadership potential?
We've replaced quorum with confidence scores.
Write throughput dropped 99.98% but the cluster is finally making informed decisions.
Irem Ergün retweeted
Yeni oyun
Schengen vize memuru ol, başvuran Türkleri eften püften sebeplerle reddet. Tepkilerine gül!
tugrulcan.com/schengen
Irem Ergün retweeted
Mathematician Terence Tao:
"we're seeing the end of the romantic era of mathematics, the age of heroes. we're moving into an industrial era where math becomes large-scale, done in big teams like the other sciences"
We're entering, rather unwillingly, a new era where exciting new things are possible, but some things are being lost
Irem Ergün retweeted
Paris, let's chat.
Come join us for a tech meetup next week, where we'll be discussing our research & engineering! Hear from our director of training integration, and speak with recruiters about life at Cohere. See you there 🇫🇷
Sign up here: luma.com/giczviut
Irem Ergün retweeted
We’ve lost an icon. Margaret Hamilton, a computer science pioneer best known for her work at MIT leading development of the flight software for the Apollo missions to the moon, has died at 90. Read about her legacy via MIT News: news.mit.edu/2026/margaret-h…
ALT Margaret Hamilton is reclined inside a NASA spacecraft module, surrounded by control panels and switches.