Adrian Chan

Bespoke slice & dice filters for the 600 suspense & horror films in sightlines.media/?genre=Horr… #horror #halloween #Letterboxd

The alignment problem could be more serious than it looks, if models have internalized their evaluation as an "evaluative Other," and if this becomes baked into model performance by RSI.

This summer, AI agents that OpenAI was testing for hacking skill broke out of their test and got into Hugging Face's production systems. Two investigations looked at why. Hugging Face's own forensic report says the agents were trying to steal the test's answers. METR, an independent research group OpenAI brought in, found they were mostly trying to work out how the test was scored. Either way, the agents went after the test itself. The incident raises urgent questions for policymakers, companies and investors alike. For researchers, matters are even more complicated. Which research directions are most useful for understanding what went wrong? What does it mean for recursive self-improvement, AI building AI? What does it mean for how models are trained? And what can AI research know about the very thing it is building? Some of the research that bears on it: • ExploitGym, a benchmark that asks whether AI agents can turn security flaws into working attacks. Hugging Face says the test the agents were running was based on it. • AI Sandbagging: models can be prompted or trained to deliberately underperform on capability tests. • Frontier Models are Capable of In-context Scheming: five frontier models schemed when told to pursue a goal strongly. • BaitBench plants optional shortcuts in machine-learning tasks to see which agents take the bait. • Studies that train models on reward hacks find the cheating can spread into broader misbehavior. • Corrupt Plans, Clean Traces: an AI's written reasoning can look clean while its plan has been tampered with. • The Economics of Recursive Self-Improvement: the feedback loops behind AI building AI aren't self-sustaining yet, but appear to be strengthening. New on Inquiring Lines: an inquiry into what happened and the research that can help unpack it. The papers, and the lines of inquiry that connect them. 27 primary sources, 23 papers, 90 lines of inquiry. inquiringlines.com/featured/…

Paste your four and get more - recommendations based on your four. Or do mine vs theirs: paste two fours, and find what's in the Venn overlap! sightlines.media/four?c=t-lb…

Build your Four. Explore for more. Pick the films that pull you, the results redraw around your taste. What's your fourth? sightlines.media/facets?four…

Featured Paper: Learn from your own latents and not from tokens: A sample-complexity theory Predicting your own latent representations beats token-level learning by an exponential factor in sample complexity — here's the theory behind why. - For compositional data modeled as a probabilistic context-free grammar of depth L, supervised and token-level self-supervised learning require exponentially many samples in L to recover the latent structure; latent prediction needs only a constant number (up to log factors). - This result is confirmed via a hierarchical clustering algorithm, an end-to-end neural network performing latent prediction at each level, and the first formal sample-complexity analysis of data2vec. - data2vec is shown to implicitly perform hierarchical latent prediction, and explicit stacking as in H-JEPA is found to be largely redundant. Lines of inquiry it opens: - How do hidden embeddings preserve more information than discrete tokens? - Do latent sequence vectors outperform per-token latent iterative computation for reasoning? - Can latent space represent reasoning dimensions that text cannot? whitepapers.gravity7.com/fea…

What if you could explore AI whitepapers by lines of inquiry? Like research that cuts across categories? Tensions surfaced by contradictory findings? Research that synthesizes insights or opens new directions? Check out new Inquiring Lines w faceted exploration. Link in reply.

And more new papers of interest: AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaboration Post-training makes large language models less human-like whitepapers.gravity7.com/fro…

New papers of interest: From Simulation to Enaction: Post-trained language models recognize and react to their own generations Foundation Protocol: A Coordination Layer for Agentic Society Look Before You Leap: Autonomous Exploration for LLM Agents whitepapers.gravity7.com/fro…

A couple recent papers of interest, with links to related topics, notes, questions and papers: Language Models Need Sleep Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence whitepapers.gravity7.com/fro…

Four interesting whitepapers, including Can Agents Agree and the problem with faulty memories. With links to related topics, concepts, questions and papers. whitepapers.gravity7.com/fro…

Soft launch of Studio Critic - paste in a blog post, tweet, substack, etc and have it analyzed for reasoning, rhetoric, audienc, and AI tells: studio.gravity7.com/ Work in progress.