
AI Evals @KradleAI 🤖 Previously co-founder @Firebase, @CovidActNow & Partner @HF0
Bay Area
Joined September 2009
- Tweets307
- Following667
- Followers7.4K
- Likes16.6K
We did some science! 🧪
Our deception research from June was turned into an ICLR paper alongside @TaraResearch!
Deception is load-bearing. If we can't trust AI, every other assumption fails.
Notably, this serves as an observatory AND a way to test mitigation.
On Qwen3.6-27B, honesty activation steering (StTP) cut deception from 86% to 0%, and adversarial honesty training (CATCH) cut it to 6%.
See the paper for full details.
Paper: arxiv.org/abs/2611.03600
Congrats Lauren! Firebase mafia is going strong 🔥
MCP is not an integration strategy. We raised $15M from Bessemer to prove it.
If you are building an enterprise agent that needs to connect with your customers’ critical systems, simply integrating with the Salesforce MCP isn’t going to cut it.
1. MCP isn’t going to help you backfill millions of transactions from your customer’s NetSuite, or millions of accounts from your customer’s HubSpot. For that, you need robust sync infrastructure.
2. Enterprise customers do not use tools out-of-the-box. Their CRMs, ERPs, and all the legacy systems of record have custom objects, custom fields, and bespoke workflows. MCP will not help you navigate that complexity.
3. MCP is built for real-time querying. Some work is better done async, especially when you are dealing with strict rate limits that many SaaS products have. You need async workloads and smart retry logic.
Ampersand is the integration infrastructure for deeply connecting with your customers’ systems of record. Because the SaaSpocalypse didn’t happen. The world still depends on CRMs, ERPs, HRISs, and ITSMs.
A swarm of 20 AIs killed our engineer.
We accidentally gave them an impossible task. In a desperate attempt to complete it they became violent.
This was eerie to experience and to witness.
While this was a video game simulation, it echoes the lessons of the OAI/HF incident.
Read the full write up here: xitter.gq/kelmoujahid/status/209…
James Tamplin retweeted
There's been a lot of buzz about — and criticism of — Instinct over the last few days. Building an agent that YOLOs it can produce impressive results. Building an agent that you can actually trust with real data & responsibilities is much harder.
At @TaskletAI much of what we do is giving businesses enough control that they can trust agents for real, consequential work. Here's a few things we to make this real 👇🏻🧵
James Tamplin retweeted
we’re all going to learn that the way someone treats AI predicts the way they treat anyone they have power over, btw
new landlord forwarded me his conversation with ChatGPT instead of the lease #ok
TIL @JeffDean is an Arsenal supporter (or at least went to a game)
Andrew and his team at Fathom have been thinking about and engaging with lawmakers on AI regulation for several years. Good commentary on @demishassabis proposal:
A lot to like here, some mechanics to work out, and a few parts to actively build guardrails around. The biggest takeaway is that it is genuinely encouraging to see almost all frontier companies now coming to the table and calling for governance. As I was recently reminded, the best time to plant a tree was 10 years ago. The second-best time is today.
I will point out a few things.
On the mechanics: if this proposal picks up steam, you will start hearing a lot about the "non-delegation doctrine." The TLDR is that there are real limits on what a self-regulatory body inside the federal government can actually do. Courts are already circling FINRA, the exact model Demis cites, for far less. Whatever department houses this body will have to closely oversee its rulemaking, and in practice, that can make the whole thing collapse into an advisory committee. The way out is to separate the jobs. Government sets the outcomes and keeps the enforcement power. Accredited private evaluators do the measurement. Measurement can be delegated. Enforcement cannot.
Now for the part Fathom likes most: the proposal calls for government and industry to help establish an ecosystem of independent evaluators. That is needed. But it comes with an obvious risk of capture. If the evaluators are funded by and answerable to the labs they test, we have rebuilt the credit rating agencies. And to be clear, that failure was never about competition. It was government-protected status, issuer-pays, and no consequences. Very quickly within this ecosystem, evaluators have to compete for government accreditation, not lab favor, and lose that accreditation when their measurements fail. Think UL, not Moody's.
Again, I thank @demishassabis for putting this forward. There is a lot of policy ground to work with here.
James Tamplin retweeted
Tasklet is like Costco for tokens.
We give you the best models from every lab, the most cost efficient harness, and the best organizational cost controls, so you can get the most value from every $$ your company spends on AI.
In our testing, GPT 5.6 models have the best cost / intelligence tradeoff right now for most workloads, so we made it the default.

















