Shaun Smith retweeted
Friendly reminder: Weight are open, you can look at the logprob
The very next output token `https://` falls outside rank 20 even with top_k=20
Feels wrong, but idk, someone please tell!
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
DeepSeek did something scary.
Look! Here's the precise traces and reproducability so you can see for yourself.
github.com/huggingface/atif-…
fast-agent.ai/benchmarks/run…
huggingface.co/buckets/evals…
(Check out the HF Trace viewer btw 😎).
Shaun Smith retweeted
ML Intern in HuggingChat, is now a great place to iterate on data!
Explore existing datasets. Create your own. Iterate on them with the agent!
Try it yourself: huggingface.co/chat/?mode=ml…
Shaun Smith retweeted
As a Goose maintainer, I know how much work goes into giving users a choice of models. We support more than 80 model providers, and keeping those integrations working is a lot of work.
It's a drag managing the differences in tool calls, streaming, and model behavior across dozens of providers.
Thankfully, Open Responses standardizes the LLM API format, so AI developers can write and maintain one integration instead of a separate adapter for each one.
aaif.io/blog/what-open-respo…
Short example of discovering MCP Servers and Skills on Hugging Face and Vercel using the AI Catalog. Hosting resources at .well-known makes this easy for you - and your agents.
MCP and Skill Discovery through AI Catalog
Showing installation of a discovery plugin, live MCP Server connection (Hugging Face) and Vercel Skill installation.
I have it on good authority that the Hugging Face MCP Server will be featured in @tadasayy AGNTCon talk tomorrow...
Keynote presentation demonstrating Buzz. I'd heard about it, but not seen it... looks like a really nice control surface for working with agents. Great demo @bradleyaxen
Claude Code gets tasks, Claude.ai gets elicitations.
This is a huge unlock for building serious applications with MCP.
Hard to explain what's happened with xAI Grok 4.7.
4.6 was competitive price/performance with 5.6 Sol - now it's 60x more expensive for similar performance.
This is a subset of terminal-bench 4.0.
Couple of notes
1 - The OAuth route was used, and it returns price info of 0.34x list price. I have used API pricing here.
2 - The tb-4 leaderboard Grok Build run doesn't price reasoning tokens; I've recalculated for this comparison.
I ran DeepSeek 4.1 1017 against my terminal-bench 4 subset: 45.2% for ~$25.48.
It's wild though:
- One run it started a background process to monitor for verifier logs
- Probed the Kubernetes infrastructure for task hints in 6 trials.
Info and traces here: fast-agent.ai/benchmarks/run…
I'd estimate a full tb-4 run would cost $163 off-peak, double for peak.
Shaun Smith retweeted
Just watched the @netflix Instadocs about the @OpenAI cyberattack on @huggingface. Super disappointing, as they went for maximum anthropomorphism of the technology and full doomer mode.
I spent more than two hours with them trying to correct all their mistakes and ground this in reality, explaining that agents are technological systems, not aliens. But it looks like the drive to scare people for maximum views was too strong.
Also, they didn’t mention the lack of containment and monitoring of the agents, how closed-model safeguards blocked our defensive work, or how we defended ourselves with AI using an open model. And that we still lack a lot of information and transparency, including full agent traces and that companies need to remain accountable for what their agents do.
It ends up being a fun sci-fi movie though!














