On a nanoGPT training-speed benchmark, we reran three published systems on new seeds. Their gains shrank due to lucky seeds. With the same coding model, the systems reach similar scores. What's different is whether you can trust the reported numbers.
New paper: Outerloop: Open and Trustworthy Multi-Agent Autoresearch. In most open autoresearch systems, the agent grades its own results. Outerloop is a kernel that runs agents as untrusted programs and keeps measurement and review outside their sandboxes.
Outerloop 0.3.0 is out: run autoresearch agents fully on your own hardware. Here, Claude Code and Codex authors run on a self-hosted GLM-5.3.
pip install outerloop-science==0.3.0
