Alessandro Favero (@alesfav) profile picture

Physics-AI fellow @Cambridge_Uni explaining the scientific principles behind AI. Formerly @EPFL, @Amazon AI.

Cambridge & London, UK
Joined September 2020
Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)Photo by Alessandro Favero (@alesfav)
AI needs vastly more data than we do. One idea might close the gap: don't predict raw signals (tokens), predict your own abstract latent representation (JEPA, data2vec). With @DanKorchinski @MatthieuWyart, on a toy model, we prove how much that helps: the gap is exponential. 🧵
14
75
7
527
53,961
Alessandro Favero retweeted
We @bold_lab_ai are hiring a large cohort of home and overseas students for Fall 2027 entry. If you’re excited about discovering the next paradigm of open-source AI and want to work on ambitious, exploratory, but scalable ML that empowers humanity, please apply by 1st of December 2026 AOE to maximise chances of funding: eng.ox.ac.uk/study/research-…
3
33
7
147
14,108
Alessandro Favero retweeted
What can you tell from an LLM response that reads [MASK] [MASK] [MASK]? Apparently enough to detect jailbreaks in masked diffusion language models! In our NeurIPS paper, we found that some jailbreaks were easier to detect using those masks than with the prompt itself. (1/6) 🧵
1
6
2
13
926
Alessandro Favero retweeted
What if an LLM could adapt its layer order to each input? What if it learned a different layer order for each classic algorithm that solves Sudoku or mazes, and you could fix that order at inference to choose which algorithm it runs? Meet LoL in under 4 minutes 🎥 (1/3)
2
13
52
2,424
Alessandro Favero retweeted
🔥 Diffusion LMs promise what autoregressive models can’t: fixing their mistakes. But new ones often change the noise and the sampler at once. Which matters? We tested it on the hypersphere. Noise: 0.8 points. Sampler: 25. We introduce Spherical Brownian Diffusion 🌐 👇🧵
16
48
7
260
15,562
Alessandro Favero retweeted
Interested in building what’s next in AI for Science? Join us at PolymathicAI in New York (NYU + Flatiron). We are looking for: Interns, Postdoctoral Researchers, Research Scientists and Research Engineers. Submit your interest here: forms.gle/EMfMccvPZ3S19BLD6
8
22
2
199
31,551
Alessandro Favero retweeted
Can a language model be based purely on self-conditioning? 🤔 Yes! Meet LAW 👋 (a.k.a. Loopholing All the Way!) 🧵1/8
10
75
12
385
38,454
Alessandro Favero retweeted
If you're not surprised, you didn't understand it…🙃 Turns out we can train generative models purely through self-conditioning! Really happy with the outcome of this collaboration! Huge thanks to @nadjushenka_, @DeschenauxAdri, @jdeschena, and the rest of the team! 🙌
Can a language model be based purely on self-conditioning? 🤔 Yes! Meet LAW 👋 (a.k.a. Loopholing All the Way!) 🧵1/8
1
6
1
39
2,710
Alessandro Favero retweeted
Two years of work, and PySR v2 is finally out! If you haven't used it before: PySR is a machine learning library that finds short equations that fit your data. I've always wanted a PyTorch for symbolic learning. PySR v2 is a big step towards that:
34
154
21
1,463
62,249
Alessandro Favero retweeted
Here is the full talk:
Had fun writing this talk for our MPhil students today: "How to avoid AI Brain Rot" Discussing how to avoid intelligence atrophy from AI use in work and study. And how to stand out on an ever-narrowing bell curve
88
1,241
81
9,715
678,223
Alessandro Favero retweeted
Whatever you make of already-clean.goodfire.com/, it has brought compositionality and generalisation back into the spotlight. I’d like to see more rigorous work testing these ideas. The Random Hierarchy Model is one example. By controlling the data's compositional structure, we recover several key features of deep learning: advantage of depth, scaling laws, linear representations and circuits. Well-motivated toy models are invaluable: they help us explain how these phenomena fit together and make falsifiable predictions about real data. Fun fact: @nickcammarata and colleagues’ earlier “Zoom In” work inspired the RHM, back when we were still thinking about cats and dogs.
2
15
124
4,102
Alessandro Favero retweeted
(1/7) Why do different languages/modalities have similar "platonic" representations in neural networks? Why does similarity peak in the middle layers of LLMs? When does it enable cross-lingual transfer? Our new paper develops a theory based on shared hierarchical structure. 🧵
2
49
4
303
11,980
Alessandro Favero retweeted
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair. It even works when the images and the captions come from different datasets. Project page: dominik-schnaus.github.io/un… ⬇️
85
360
85
2,409
248,283
Alessandro Favero retweeted
The Bitter Lesson for harness design and formal math! Instead of hand-designing how an AI should prove (decompose, plan, assign agents), what if we just give a frontier model good Lean feedback and let it keep working? We built h1. TL;DR: → SOTA on formal theorem proving benchmarks → can solve open conjectures → matches/beats hand-designed workflows → one long attempt can beat many retries [1/7]
17
40
11
289
36,213
Alessandro Favero retweeted
A multilingual LLM encodes a sentence and its translation almost identically - the “platonic representation hypothesis.” Observed, never explained. The facts pile up: similarity peaks in the middle layers (not the ends), coexists with language-specific structure, and grows with language proximity, model quality, and data. We explain all of them -and predict something new: remove from each layer the part linearly predictable from the previous one, and cross-lingual similarity rises. Confirmed in real LLMs.
6
18
1
92
4,487
Alessandro Favero retweeted
Say hi to PUMBA! 🐗 Masked diffusion language models are trained one way and sampled another. Our new paper studies how to make MDM training more aware of the trajectories followed at inference, and how this can lead to more efficient text generation. Paper link below 🧵
2
24
66
2,529
Alessandro Favero retweeted
Mislabeled 3D protein structures risk polluting public scientific databases and misleading critical downstream research. So today in @nature, we're announcing SynthID Bio to watermark generated protein sequences and structures. deepmind.google/blog/introdu…
1
3
14
527
Alessandro Favero retweeted
🔺Happy to share a project done during my internship🔻 It was amazing to collaborate 🥹 @ValentinDeBort1 @ArnaudDoucet1 @agalashov @AndrewC_ML @JamesTThorn and Kevin Li (who I cannot find on Twitter)
🔥Welcome 🔺Simplex Diffusion Models🔻! Standard discrete diffusion models suffer from "information collapse" because they discard uncertainty at intermediate steps by sampling categorical tokens. (1/5)
4
27
87
6,277
Alessandro Favero retweeted
We are having our first @SimonsFdn yearly meeting on the Physics of Learning and Neural Computation Sept 24th-25th and all are welcome to join on this zoom: simonsfoundation.zoom.us/j/9… We have an excellent set of speakers. Here is the agenda: simonsfoundation.org/event/s… Join our webinar starting Sept 24th 9:30am ET to learn how the structure of data, architectures, and learning dynamics conspire to empower AI to learn, scale, compute, imagine, reason and collaborate! See physicsoflearning.org/ for more about our collaboration.
2
58
11
285
43,728