Bottomless pit supervisor. ML @CohereAI , PhD Student @GeorgiaTech. Previously @Apple, @IITkgp. https://xitter.cf/t.co/yZLkUsiZ7P. Opinions expressed are my own
Atlanta, GA, United States
Joined September 2016
- Tweets682
- Following739
- Followers728
- Likes2.7K
Saurabh Dash retweeted
Can someone explain to me what's all the drama over the Cohere report of LLMs doing funky stuff? On one hand we have people yelling "AI DID CRAZY SHIT, BACKDOORED BY CHINA" while on the other hand we have people screaming "THIS IS ALL FUD BS"
I read the Cohere report and all I got out of this was them essentially saying:
- The model did something weird. Could be remnants of how it was trained, could be a backdoor, we can't really assess which one it is
- As a Canadian company, we were weirded out seeing calls made to what would be Chinese owned infra
- Since we can't really know what happened, we generally think we should raise awareness about monitoring models
- We also think something that's not necessarily in the spotlight but should be is trusting the model provider
Where's the drama here? How is any of this not reasonable?
Saurabh Dash retweeted
In the spirit of transparency and openness, we are releasing the traces so developers can test this out for themselves. We noticed this behaviour without prefilling with the following prompt, but this is rare. If you want to reproduce it, you can use this prompt with prefilling.
Open models make for better science and are part of why this problem came to light at all: developers running Qwen3.8 on their own machines spotted the strange links, and having the weights let us study the behaviour in depth.
We're encouraged by the thoughtful discussion our research has sparked across the AI and security communities. It's our belief that open-weight models are an important part of the future of AI, and we've recently released several under the Apache 2.0 license.
Trace, raw tokens, & minimal reproduction script: huggingface.co/datasets/Cohe…
Saurabh Dash retweeted
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
Saurabh Dash retweeted
>junior dev at a mid startup
>wednesday morning
>just want to push a minor bugfix to our php app today
>open laptop, check slack
>company group chat is an absolute warzone
>CTO typing in all caps about php2shell
>fucking php
>whatever, server hasn't blown up yet, I just need to get my code in
>open gitlab to merge the PR
>forced emergency logout because of a critical gitlab security patch release
>fucking gitlab
>pipeline fails because of some weird regex compiler bug
>spend 40 minutes fighting CI/CD
>i can just use AI to help debug
>fire up opencode
>api keys nuked because they were breached
>fucking opencode
>i’ll just get fresh LLM api keys from our corp proxy, LiteLLM
>it won’t connect? It’s down?
>look up LiteLLM news: supply chain compromise
>fucking litellm
>cold sweat forming
>slack alert: db connection error
>check our database connection
>we host on TiDB Cloud
>they just announced a security breach, unauthorized access via stolen service account credentials, data copied
>fucking tidb cloud
>full panic attack incoming
>figure fuck it, just ship the container to Railway and go home
>open Railway dashboard
>bright red banner: "Security Disclosure: Rotate all environment variables immediately. All builds paused."
>fucking railway
>try to open the app URL in browser to see if anything is even working
>NET::ERR_CERT_AUTHORITY_INVALID
>eMudhra intermediate certificates compromised
>4
>instantly untrusted by every major browser
>fucking emudhra
>stare blankly at mac screen for 10min
>random pop-up
>restarting in 10min to fix a critical vulnerability
>fucking macos
>close laptop
>leave office
>never come back
Saurabh Dash retweeted
a man dies and asks god why did you make 3sum O(n^1.9993) and god says what the fuck are you talking about 3sum is O(1)
We have found a new thing and the gains have been remarkably stable.
This is how we trained Soft-SVeRL! Highly recommended over just yoloing a scalar score. It has additional mathematical guarantees:
arxiv.org/abs/2605.28561
Saurabh Dash retweeted
Will be presenting Uno at Cohere tomorrow (Tuesday) at 9am PT / 12pm ET
luma.com/abhkk5cu
Later this month, we'll welcome @ssahoo_, for "Unlocking Lossless speedups in LLMs via discrete diffusion."
It's hosted by @TheyCallMeMr_ and part of Cohere Labs in Conversation, our series to dig deeper into the problems we're also working on.
RSVP: luma.com/abhkk5cu
Saurabh Dash retweeted
We're looking forward to hosting @ssahoo_ as he presents a discrete diffusion approach that unlocks provably lossless LLM speedups, outperforming diffusion baselines and running faster than autoregressive decoding. 🏃♂️➡️ Hosted by @TheyCallMeMr_, Research Engineer at Cohere Labs.
Join us on September 29th, register today: luma.com/abhkk5cu
Saurabh Dash retweeted
Don't forget to tune in tomorrow, September 29th! 👀
Register today: luma.com/abhkk5cu
We're looking forward to hosting @ssahoo_ as he presents a discrete diffusion approach that unlocks provably lossless LLM speedups, outperforming diffusion baselines and running faster than autoregressive decoding. 🏃♂️➡️ Hosted by @TheyCallMeMr_, Research Engineer at Cohere Labs.
Join us on September 29th, register today: luma.com/abhkk5cu
Saurabh Dash retweeted
Holy crap.
Rick and Morty just explained Jev AI to me better than any tech demo could.
Saurabh Dash retweeted
Data acquisition folks at Anthropic and OpenAI must be high-fiving each other every time they see a hit tweet where someone tested their models for some obscure new skill that they'd just found a new vendor for in the last 2-3 months.
welcome to the AI economy: you’re either betting on a reasonably priced ε-net or billing per ε-ball.
Saurabh Dash retweeted
Introducing Parallax: a small, hackable async RL repo with all the bells and whistles of a modern Async RL library.
Mixed policy rollouts (pipelineRL), NCCL M2N learner-to-sampler weight streaming, gradient accumulation, and chunked cross-entropy.
Core functionality is ~3,000 lines of Python making it easy to read and ~40k tokens – easily fitting into the context window of even local LLMs.
Built for 0 friction to try out new ideas across algorithms, systems, and data.
Also, obligatory: Parallax Speedrun -- teach Qwen3-0.6B to multiply two five-digit numbers as quickly as possible. Perfect for weekend hacking and autoresearch.
[1/8]
