PhD Student- Complex Networks & Systems and Cognitive Science, Indiana University
Joined July 2019
- Tweets214
- Following269
- Followers89
- Likes790
The murder of Charlie Kirk is a tragedy. I’m deeply saddened by this. We need to learn how to talk to each other again.
Jacob C Tanner retweeted
The Lord of the Rings does not take place on an imaginary planet — it's Earth.
Middle-earth is our forgotten past, before recorded history, when Eden (Valinor) was a real place.
The truth of Tolkien's world will blow your mind... 🧵
Jacob C Tanner retweeted
New paper! “In-Context Learning of Representations”
What happens to an LLM’s internal representations in the large context limit?
We find that LLMs form “in-context representations” to match the structure of the task given in context!
1/n
Jacob C Tanner retweeted
The fall of Rome is widely misunderstood.
It wasn't invasion, disease or famine that truly brought it to its knees.
Rome collapsed because the birth rate did… (thread) 🧵
Jacob C Tanner retweeted
LOTR: THE FELLOWSHIP OF THE RING was released 23 years ago this week. An adaptation of Tolkien’s classic novel, and the first entry in Peter Jackson’s The Lord of the Rings trilogy, the story of how it was made is proof that one does not simply walk into Mordor…
1/76
Jacob C Tanner retweeted
Review "metastability demystified" is finally out @SpringerNature @NatRevNeurosci (rdcu.be/d3bh9), led by @FranHancock1 & @_fernando_rosas w/ contributions from me & colleagues of many distinct perspectives. We identify the converging mechanisms & common misconceptions.
Jacob C Tanner retweeted
The apparent "philosophical problems" of quantum mechanics are not unique to QM at all: they are in fact the same problems that arise whenever one attempts to construct an abstract model of reality. We can see these problems already in high school-level mechanics. (1/14)
Jacob C Tanner retweeted
Very interesting! how does it relate to this work, studying phase transitions in the dynamics of diffusion? nature.com/articles/s41467-0…
Jacob C Tanner retweeted
Past societies produced so much beauty because they knew that math and beauty are deeply connected.
It all started when Pythagoras discovered something mind-blowing about reality:
The universe is not made of matter — but music... (thread) 🧵
Jacob C Tanner retweeted
(1/5) Very excited to announce the publication of Bayesian Models of Cognition: Reverse Engineering the Mind. More than a decade in the making, it's a big (600+ pages) beautiful book covering both the basics and recent work: mitpress.mit.edu/97802620494…
Jacob C Tanner retweeted
Doubling o1-preview performance on ARC-AGI with one simple trick 🚀
tldr: by providing human-like representations to o1, we are able to substantially increase performance on @arcprize.
Jacob C Tanner retweeted
📢Thrilled to introduce the #VirtualLab: a team of AI scientist agents (AI chemist, AI reviewer...). Virtual Lab is led by an AI professor w/ feedback from human scientist.
The Lab created new nanobodies that we experimentally validated to bind to recent #covid variants🚀🧵
Jacob C Tanner retweeted
I compiled all the emails released as part of the Musk v. Altman lawsuit in chronological order (link in reply).
IMO a really valuable read. Extremely consequential decisions made in these emails.
Jacob C Tanner retweeted
If you are interested in Generative #AI, or statistical physics, you will know that you can use latent diffusion models to make synthetic images (or videos), but these methods are a bit slow (I explain here how the Fokker-Planck formulation and Langevin diffusion are related):
youtube.com/watch?v=fOORfzGj…
About an hour ago some Russian scientists from Skoltech posted a quite considerable leap forward in this field, computing these maps much faster + in one step:
arxiv.org/pdf/2403.13117
ALT Top picture from the Skoltech paper out tonight - https://arxiv.org/pdf/2403.13117 Lower image I made about 20 years ago - https://www.researchgate.net/figure/Deformation-based-on-a-uniform-grid-Thanks-to-Paul-Thompson-for-this-image_fig1_45189084
"This is just a stock image of a cat. Physicists are not allowed to own cats, for various reasons." (from a great lecture [1] on generative AI [2] by Jordan Cotler)
[1] youtube.com/watch?v=bHLdAFLB…
[2] This is a very clear explanation of denoising diffusion probabilistic models (DDPMs [3]), and how they arise in statistical physics (diffusion of smoke in air or ink in a fluid, drifts of sand dunes in the wind).
[3] DDPMs and their compressed extension, LDMs, are used in text-based image generation (MidJourney, SORA, Kling). A trickier but very clear lecture is the one by the inventor of DDPMs, Jascha Sohl-Dickstein, also featuring his dog (!)
youtube.com/watch?v=XCUlnHP1…
[4] for an even deeper version, someone has posted an entire book on Twitter tonight, which is pretty amazing "Statistical Optimal Transport": arxiv.org/abs/2407.18163 #AI #DDPM
ALT Schrödinger's cat. This is a pretty good joke as the probability of the cat being in the image also depends on the probability of the image under the flow. :)
Jacob C Tanner retweeted
“We’re very excited to see what people do with this”
AlphaFold3 is open at last go.nature.com/4fHAfio
Jacob C Tanner retweeted
🚨 So, why do we need weight decay in modern deep learning? 🚨
The camera-ready version of our NeurIPS 2024 paper is now on arXiv (a major update compared to the first version).
Weight decay is traditionally viewed as a regularization method, but its effect in the overtraining regime is quite subtle and its interaction with the implicit regularization effect of SGD plays a crucial role.
In the undertraining regime (e.g., in LLM pretraining), however, the effect of weight decay is totally different: it sets an implicit learning rate schedule for AdamW and enables stable training with bfloat16 precision. This explains why weight decay is still widely used for LLM training with standard optimizers, such as AdamW.
This is joint work with @dngfra, @adityavardhanv, @tml_lab.
Jacob C Tanner retweeted
🌟 Cai Lab @Nature paper alert! In new work led by @mysteriousjoe_, we find that rest periods after learning not only stabilize new memories BUT ALSO integrate new memories with older ones from days past! (1/9)
Read it here: nature.com/articles/s41586-0…
Jacob C Tanner retweeted
Jacob C Tanner retweeted
There are many things we don’t understand about deep learning. Our new NeurIPS paper (w/ @AliciaCurth) makes the mistake of trying to tackle too many of them 😅
A simplified model of deep learning describes double descent, grokking, gradient boosting & linear mode connectivity🧵
