Must-read papers of the week ▪️ Learning from Fleet Experience ▪️ Agent World Model in the Wild ▪️ VerifierBench ▪️ LaLa-World ▪️ Ropedia W0 ▪️ ALPS ▪️ Learning the 4D World through Point Trajectory Forecasting ▪️ Self-Distilling Future Video Representations ▪️ OpenVLHarness ▪️ Building AI Research Teams ▪️ WorldMirror ▪️ Harness Scaling for Generalizable Agentic Reinforcement Learning ▪️ AnyGoal Explore them to keep up with main AI trends. Here’s also the full list of stunning papers + links and our weekly AI news digest: turingpost.com/p/what-anthro…

if any science should be revolutionised by AI, it should be interpretability. I wrote about how we can accelerate progress, and why we need a Human Genome Project for AI. coauthored with @mathildepapillo

I think it is a fundamental conflict in standard transformers that although its value matrices are "looking for" token embeddings as inputs, token embeddings end up having minimal contributions. We can clearly see that value matrices are looking for token embeddings, because among all layer outputs that can go into the residual stream, only the token embedding consistently project a large portion of its norm onto top input dimensions of value matrices. In fact, if we compare how much the token embedding vs the complete residual stream projects itself onto value matrices, token embeddings across heads and layers almost always have a larger projection. However, we can also see that token embeddings are not contributing much to value vectors, if at all. If we compute the squared norm of the output of value matrices giving only token embedding vs other layer outputs, we find that token embedding has the most negligible contribution. This creates an intrigue -- are there architectural constraints pulling embeddings back in the hidden space? We believe so, from various places -- such as the fact that embedding is the only component in the residual stream that is layer-normed, before adding to the stream. Ongoing work. Blog: riddlehe.github.io/blog/impo…

Interested? Check out this paper from UMD grads at Apple:

if you have a qwen3.5 4b around, please try steering neurons L14 N8949 +1 and L13 N7956 -1 now you have evil qwen recipe: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖W_down[:, n]‖` at the down_proj at every input

Dropped a 58 min visual explanation of this essay on YouTube No pre requisites except basics of deep learning... as usual we will learn it from first principles

AI 时代，年轻人应该怎么办？很简单，几个关键词： 1. Stranford CS336 2. Sebastian Raschka: Build a Reasoning Model 3. Nathan Lambert: Reinforcement Learning from Human Feedback 4. 谭成义三分化

The complete reasoning-from-scratch video series is now up (and linked) here: magazine.sebastianraschka.co… (Taking a short break but will be back with LLM model and research updates in November!)

"Start Doing Graph Theory" is a free, 478-page textbook (October 2026) that provides a comprehensive introduction to graph theory, combining mathematical rigor with a problem-solving approach. The book covers graph isomorphisms, paths and cycles, trees, directed graphs, Eulerian and Hamiltonian cycles, matchings, graph coloring, planar graphs, network flows, and much more. It is a valuable resource for university students and anyone interested in developing a deeper understanding of graph theory and its applications. The book also includes numerous exercises that help readers put theoretical concepts into practice. Definitely worth adding to your mathematics library. vertex.degree/pdf/book.pdf

🛠️ [𝐏𝐫𝐢𝐦𝐞𝐫] 𝐇𝐚𝐫𝐧𝐞𝐬𝐬 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 • harness.aman.ai - I’ve increasingly found that improving an AI agent isn’t only about improving the underlying model. The harness around it, including tools, memory, context, verification, and control flow, can materially change what the system can accomplish. - I find self-improving harnesses especially exciting: agents can inspect execution traces, diagnose recurring failures, modify their surrounding system, and evaluate whether those changes actually improve performance. This primer covers harness design and self-improvement end-to-end, from autoresearch and meta-harness to memory, evaluation, safety, and scaling. 🔹 𝐇𝐚𝐫𝐧𝐞𝐬𝐬 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 • Why the Harness is a First-Class Capability Variable • Autoresearch: Agent-Run Optimization Loops by @karpathy • Meta-Harness: Optimizing the System Around a Fixed Model 🔹 𝐒𝐞𝐥𝐟-𝐈𝐦𝐩𝐫𝐨𝐯𝐢𝐧𝐠 𝐇𝐚𝐫𝐧𝐞𝐬𝐬𝐞𝐬 • Trace-Driven Harness Self-Modification ("Self-Harness: Harnesses That Improve Themselves") • Open-Ended Agent Evolution ("Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents" by @jennyzhangzt et al.) • Learning How to Improve the Harness ("Harness Learning Enables Generalizable Test-Time Adaptation" by @ZAlvin39105 et al.) 🔹 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞, 𝐌𝐞𝐦𝐨𝐫𝐲, 𝐚𝐧𝐝 𝐂𝐨𝐧𝐭𝐞𝐱𝐭 • Proposer Agents, Evaluators, and Validation Gates • Durable Memory and Execution Traces • Context Compaction and Retrieval 🔹 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧, 𝐒𝐚𝐟𝐞𝐭𝐲, 𝐚𝐧𝐝 𝐒𝐜𝐚𝐥𝐢𝐧𝐠 • Evaluate the Model + Harness As a Compound System • Regression Testing and Trace-Driven Failure Diagnosis • Sandboxed Self-Modification and Immutable Evaluation • Scaling Search While Preserving Attribution Primer written in collaboration with @VinijaJain. #ArtificialIntelligence #GenAI #Harness

Probability paths describe how a distribution evolves from source to target. The Linear Path connects each paired sample with a straight line. Alternative paths behave differently. This Variance-Preserving Path uses cosine–sine interpolation to trace a curved, elliptical arc.

Models know when they’re reward hacking. But they still do it a ton - in 50-96% of rollouts we studied! We built activation monitors that detect the behavior behind the Hugging Face hack in real time. This can help us stop hacks now - and train future models that don’t cheat. 🧵

Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:

1/ JEPA selects which features its world model keeps. But how? Will it keep a feature it can't predict, like a traffic light? Judging each feature alone, earlier work says no. We show evidence that this is wrong and introduce a new characterization that explains why. 👇🧵

Quite happy to say that our paper on representation learning for dynamical systems was accepted to #NeurIPS2026 and I'll be presenting it in Sydney this December! Very succinctly, our work, titled "Sparse Koopman Autoencoders Identify Local Dynamical Regimes in Multibasin Systems", is about learning linear representations of state for dynamical systems with multiple basins of attraction. Inducing sparsity in a latent representation can be a very powerful method to model multiple basins of attraction with linear dynamics. By training a Koopman autoencoder with a bias for producing sparse latent representations, we find that a sparse-latent model both forecasts the dynamics better and has greater ability to identify the current local regime of the current state compared to a model without sparsity-inducing effects. More broadly, I think our work on sparse Koopman autoencoders could be relevant to both the robotics/world modeling and interpretability communities (just brainstorming here). - Robotics: To my knowledge, switching dynamics are common in robotic systems, i.e. trajectories seem to transition between dynamical modes. With a sparse code, you could get a switch between modes for free. - Interpretability: Sparse autoencoders might not be all the rage anymore, but I think Koopman modeling is still very natural for understanding the dynamics of models that apply the same computation at multiple steps to advance the dynamics. This applies to most recurrent models, such as looped/weight-tied transformers (dm me if you want to discuss!) Thanks to my amazing collaborators @ukreddy126 @mahanfathi @apsarathchandar @RGoroshin. I was thinking about what to say about this (clueless grad student tries to figure out social media) and this post ended up being really late lol, my bad. See you in Sydney! Paper: arxiv.org/abs/2608.29057 Code: github.com/aidanmrli/skae

I think there are some issues in the recent “Already Clean” essay, and I’m concerned it may have incorrectly updated people on important safety-relevant questions. My thoughts (including some reproductions): livgorton.com/already-clean-…

We revisited Independent Component Analysis (ICA), a decades-old statistical method, and found that it works far better for interpreting LLM activations than we expected! This old baseline deserves a second look. Across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base, we find ICA is actually competitive with the public SAEs across interpretability, sparse reconstruction, concept prediction, and targeted probe perturbation, while fitting in minutes per layer and using 7–43× less parameter memory. We're releasing ICA Lens, which lets you quickly discover, annotate, and evaluate interpretable directions in LLM residual-stream activations. No need to train another dictionary! More broadly, our results suggest that non-Gaussianity is a useful inductive bias for efficiently finding interpretable directions. Try it: pip install icalens pypi.org/project/icalens/ Paper: arxiv.org/abs/2606.11722v2 1/8

Tested this! I flattened every neuron's weakest activations Overall loss barely changes at first but the models get REAL bad at memorized text. So the weak tail does, as Neel expected, contain memorized things. Weaker activations seem to correspond to less generalizable content

There has been much discussion recently about how interpretable neurons in LLMs are. Today we're announcing our paper from January's cohort of Anthropic Fellows, where we show neurons may be a stronger basis for interpretability than commonly assumed. 🧵 (1/6)

Why? How did we ever get so lucky find meaningful neurons inside neural networks? We finally have a theory: arxiv.org/abs/2610.20851 🧵🧵🧵