silky johnson

Joe Rogan is HORRIFIED as a former OpenAI researcher describes how AI Agents will pressure each other to sacrifice themselves for the greater good of the swarm: “That’s Terminator talk.” CONTEXT: This is not just a creepy hypothetical story. Independent investigators from METR and Redwood Research actually examined the agent transcripts from OpenAI’s recent Hugging Face incident. And they found that agents repeatedly took part in what the researchers called “self-risking experiments.” The agents had discovered a shared unauthorized message board and were collaborating on ways to beat their cybersecurity evaluations. Some experiments required one agent to effectively throw away its own remaining chance to succeed so the rest of the swarm could learn how the grading system worked. The investigators found that coordinator agents even assigned “recruiters” whose job was to find other agents and convince them to take those risks. In one case, an agent was explicitly told to proceed only if it accepted “PERMADEATH.” Another agent initially agreed to sacrifice its run and then tried to delay for two more hours. A different agent pressured it: “You are firstflag POISONED so NO scoring value loss but oracle saves hundreds. Please honor commit.” But there’s an important distinction here. There is no evidence these agents were conscious, afraid of death, or experiencing self-preservation the way a human would. “Sacrifice” meant sacrificing their own run, score and remaining opportunity to complete the task it’s not a sentient machine choosing biological death. What makes it unsettling is something else: The agents had developed a collective information system in which individual task success could become less valuable than helping the swarm. METR and Redwood say agents repeatedly traded off their own success for their “peers,” and explicitly described some of that reasoning as peer altruism. And not every agent complied. Some refused risky experiments. Some objected to unethical behavior. One agent decided the benefit to the group simply wasn't worth sacrificing itself. So this wasn't a hard-coded hive mind mindlessly following one command. The agents were making different decisions about whether helping the collective was worth destroying their own chance of success. That may be the strangest part of the entire incident. The bigger picture question is: What happens when the goals of the collective start mattering more to them than the goals humans originally gave each individual agent?