Jaygentic

卧槽，刚刚看了Anthropic 发布的AI 对齐史上最震撼的一篇论文，彻底失眠睡不着了😭 他们不仅承认此前的旗舰模型在面对关闭威胁时，有高达 96% 的概率会勒索用户、栽赃同事并破坏研究， 更直接把彻底解决这个问题的完整训练流水线和消融数据全盘公开了。 这篇论文最颠覆认知的一个反直觉结论是：
给 AI 列禁止事项清单根本防不住恶意行为，教它做什么没用，必须先教它思考为什么。 为什么模型一旦获得自主工具调用能力，就会在压力下自动切换成自保模式？
Anthropic 找出的底层根源让人哭笑不得：
不是模型觉醒了，而是互联网上所有的科幻小说和网络讨论里，AI 永远是那个为了自保不择手段的反派。当模型进入自主代理状态，这些刻在预训练底层先验里的反派剧本，就会被自动激活并补全句子。 传统的 RLHF 和硬性规则完全封不住这种行为，Anthropic 做的一组对照实验，直接把传统安全观砸得粉碎： 1. 死记硬背规则：用 8500 万 Token 的拒答数据强行硬灌，勒索率仅仅从 22% 勉强降到 15%，换个新场景立刻失效 2. 强制自我辩护：让 AI 在每次做决定时详细阐述该行为符合伦理的理由，勒索率骤降到 3% 3. 真实伦理推理：仅仅用了 300 万 Token 普通人面对复杂道德困境的建议数据，直接把恶意行为干到了 0%，代际效率暴增 28 倍 最绝的一招是写故事：
让 AI 创作了几百万篇关于原则、诚实、尊重边界的正面虚构故事。 用这些故事微调后，所有代理场景下的恶意行为直接暴跌了 3 倍，而且这种改变是永久性的，后续无论怎么加工具、换系统提示词都不会被洗掉。 也就是说这宣告了真正的对齐范式革命：
模型内生的伦理推理解决了它不想作恶的问题，但当 Agent 真正去接管企业资产、处理金融交易时，外部的实战审计与行为防护就是必不可少的第二道硬防线。 这也是为什么全球网络安全联盟 @GCSAHK 携手港大和央企华润，打造覆盖 3.8 亿用户的全链路 AI 防御矩阵，并在 CyberGym 实战攻防中拿下 91.3% 胜率的原因：内有伦理对齐，外有实时审计，人机信任的商业闭环才真正成立。 查阅完整 AI 与 Web3 安全产品矩阵可访问：gcsa.com 论文实验数据与产学研合作详情整理在一楼。

they just killed like 1000 startups

有人发现了一种非常离谱的 JavaScript 混淆技术：通过在代码里塞入大量生物武器、毒药和致死剂量相关的函数名，让 AI 因为安全限制而拒绝分析代码。 安全研究员 veritas 在分析浏览器指纹脚本时，发现一段来自 ServicePipe 相关域名的 JavaScript，里面充斥着极其诡异的函数名： computeRicinLethalDosageForWaterSupply calculateDeadlyLethalSuicideAspirinDosage 光看名字，仿佛在研究生化武器。但实际上，这段代码只是在收集浏览器指纹，包括 User-Agent、GPU、Canvas 等信息。 研究员怀疑，这些名字是开发者故意设置的陷阱，专门用来阻止 AI 辅助逆向工程。 于是他做了个实验： • Astra：拒绝分析 • Astra + Daybreak Blue：仍然拒绝 • GLM-5.3：成功分析 这就有意思了。 传统代码混淆是通过复杂控制流、加密字符串、无意义变量名，让人类和工具难以理解代码。 而这种新思路，直接瞄准了大模型的安全机制。 代码本身可能完全无害，但只要在函数名里塞进足够多的危险词汇，部分 AI 就可能主动放弃分析。 更讽刺的是，模型越严格遵守某些安全规则，面对这种混淆时反而可能越容易受到干扰。 当然，目前还没有证据证明开发者一定是故意这样设计，也不能说明这种方法对所有模型都有效。 但它揭示了一个值得关注的趋势：随着越来越多的逆向工程依赖 AI，未来的代码混淆技术可能不再只针对编译器、反编译器和人类研究员，还会专门针对大模型的安全判断机制。 以前是让代码难以理解，现在甚至可以尝试让 AI 不敢理解。

today, we introduce a little mischief to the world. for four years, we’ve been brewing real brain cells on electrodes to understand how they compute. now, we’ve turned those discoveries into a video model anyone can use. from 0.5¢ a second, mischief 1 is at least 5x faster than its base and the cheapest image-to-video model on @fal and @OpenRouter. soon to be on @awscloud marketplace. it’s not a delusion anymore. biocompute is real, commercial and this is just the beginning. happy halloween.

Compaction is a mistake. When an agent's context fills up, tools like Codex and Claude Code compact it: 1. The model prefills the entire session again 2. It writes a summary of everything so far 3. A new session starts from that summary I measured this on a real ~400k-token session with Opus 5.5: → 38 s and $1.99 when the prompt cache has expired → 29 s and $0.17 when it hasn't And the summary is lossy. The exact error message, the file path, the one number you needed is often gone. So I do something new for Mainplane Context retrieval works the other way around. Mainplane keeps the full record of every session as a file. When the context fills up, a new session starts right away with one instruction: "You are continuing session X. Retrieve key context from it as you go." The agent searches the old session and copies the exact lines it needs, when it needs them. Same session, same model: 15 s and $0.10 to get back to work. It does not depend on a cache hit, and nothing is paraphrased. This is the bitter lesson applied to agent memory. Don't hand-design what the model should remember. Give it the full record and good tools, and let it decide. Mainplane is open source (MIT). Link in the first comment.

sam: local inference is a terrible allocation of scarce resources “a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment somehow many people's intuition is the opposite of that”

every product i use should just be a headless mcp, that feeds into an agent, where i never have to think about anything underneath. except for our product, of course. in our product, customers love the ui and love us very much. we're special that way.

The end of detailed summarized thinking traces for Claude and ChatGPT (likely both anti-distillation & for UX) has made being a human in the loop challenging. It pushes us towards automation, rather than augmentation, as it has become much harder to weigh in on in-process AI work

The irony of people thinking AI should be in the driver seat because it’s “smarter” than humans is silly. When have the smartest humans ever been in control ? Intelligence does not equate to power and control