We developed a straightforward method of combining confidence-weighted judgments for any number of humans and AIs. w @yanezlang, Omar Minero, @ken_lxl 2/2

New blog, "Backwards Compatible: The Strange Math Behind Word Order in AI" w @ken_lxl. It turns out the language learning problem is the same for any word order, but is that true in practice for large language models? paper: arxiv.org/abs/2505.08739 BLOG: bradlove.org/blog/prob-llm-c…

Finally, LLMs can be augmented with neuroscience knowledge for better performance. We tuned Mistral on 20 years of the neuroscience literature using LoRA. The tuned model, which we refer to as BrainGPT, performed better on BrainBench. 7/8

In the Nature HB paper, both human experts and LLMs were well calibrated - when they were more certain of their decisions, they were more likely to be correct. Calibration is beneficial for human-machine teaming. 5/8

All 15 LLMs considered crushed human experts at BrainBench's predictive task. LLMs correctly predicted neuroscience results (across all sub areas) dramatically better than human experts, including those with decades of experience. 3/8

To test, we created BrainBench, a forward-looking benchmark that stresses prediction over retrieval of facts, avoiding LLM's "hallucination" issue. The task was to predict which version of a Journal of Neuroscience abstract gave the actual result. 2/8

"Large language models surpass human experts in predicting neuroscience results" w @ken_lxl and BrainGPT.org. LLMs integrate a noisy yet interrelated scientific literature to forecast outcomes. nature.com/articles/s41562-0… 1/8

"Beyond Human-Like Processing: Large Language Models Perform Equivalently on Forward and Backward Scientific Text" Our take is that large language models (LLMs) are neither stochastic parrots nor faithful models of human language processing. arxiv.org/abs/2411.11061 1/2

"The Dimensions of dimensionality" is out in TiCS w @BDRoads We hope to interest those new to embedding approaches and pros. Besides serving as a pracitcal guide, you can pwn your colleagues when they report dimensionality alone as a meaningful result. doi.org/10.1016/j.tics.2024.…

Andrea Hiott and I had a fun conversation on her Love & Philosophy, Beyond Dichotomy podcast. We got into how AI is affecting neuroscience and society broadly; where it's all going and how it started. Bonus random takes on how are basically ants :) youtu.be/EA44JEJPrc0?si=a9Kg…