Gregor Stewart (@olakrez) profile picture
Joined November 2008
With some probability, every code change introduces a new issue. If we're making very many changes, that probability had better be very low. I think it can be, but I don't think it is.
In a world where a non-trivial number of these outages were caused by bad patches and 'agentic mishaps', what's the conversation we are having with the frontier corporations about the quality of their tools and their reesponsibility for those outcomes? There's no question that these models can find vulnerabilities, it's not a given nor has it been established under what parameters they're reliable at patching–– everywhere from: does the patch effectively cover the bug or just a narrow implementation? does it have other systemic consequences? is the patch introducing new bugs? Take some time to sit with that final question. Because if we can't establish that AI-generated patches reliably increase the integrity of the code they're modifying, then what they've effectively encouraged everyone to do is to increase the entropy in all of these codebases and instability is hardly an unpredictable outcome in that.
12
Manchurian Qwendidate!
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected. Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions. Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent. But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models? We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem. Read the full report: cohere.com/blog/concerning-b…
1
2
889
I'd forgotten that The Informant opens with several minutes of vintage Nagra pornography.
36