Timothée Lesort (@TLesort) profile picture

Researcher at Imec AI Labs. Continual Learning Vision/LLMs. Prev. Postdoc. @Mila_Quebec, PhD @IP_Paris_

Berlin
Joined February 2015
Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)Photo by Timothée Lesort (@TLesort)
Timothée Lesort retweeted
I recently moved to TU Darmstadt (Germany) and am currently hiring PhD student through ELISS. ellis.eu/news/ellis-phd-prog… If you interested, please apply and choose me as a potential advisor. Topics include Continual/Distributed/Federated/Active/Transparent AI.
8
35
2
217
10,441
Timothée Lesort retweeted
So happy to highlight one of the many applications of Botanic-1, our latest plant FM, on @googlegemma 's website ! xitter.gq/GoogleAI/status/210715…
2
8
2
21
1,753
Timothée Lesort retweeted
Excited to announce our @COLM_conf workshop "Context Beyond the Window"! We have a stacked lineup of speakers focusing on one of the most important topics today: context management. Venue is non-archival; We welcome all relevant 4 or 8-page submissions! xitter.gq/dvnxmvl_hdf5/status/20…
🚨Excited to announce our workshop Context Beyond the Window hosted at COLM in SF! 🚨 LLMs have finite context windows, yet real-world tasks demand absorbing, retaining, and acting on information that far exceeds any single prompt. 1/3 We're looking for submissions across: context-beyond-window.github… • Context compression 🧃 — token compaction, recursive subagent calls, and external memory for storing and retrieving information • Efficient architectures 🚀 — sub-quadratic attention variants that make extremely long context computationally feasible • Continual training 🌱 — test-time training on streaming data, context distillation, and knowledge accumulation through continued pre-training • Agentic memory systems 🐘 — scaffolds and test-time scaling techniques that improve knowledge retention and acquisition in LLMs • Evaluation 🎯 — benchmarking models on increasingly long-horizon tasks
3
14
1,149
Timothée Lesort retweeted
The DiLoCo team at Google DeepMind and Google Research is proud to release Decoupled DiLoCo, the next frontier for resilient AI pre-training. Decoupled DiLoCo enables training with datacenters across the world, using heterogeneous hardware, and never halting the system despite hardware failures.
35
86
33
629
2,742,770
Timothée Lesort retweeted
(continual) pre-training is not dead! some thoughts about cost per task (on cursorbench) being 2x lower, imo it can be due to: - new base model: seems straightforward but it's not imo, you need to optimize the inference/training stack (both for rl and consumer inference), GLM-5 > V3.2 doesn't mean glm-5-base > V3, they may not be equally malleable to post/mid training - more optimized kernel and inference stack/serving (most likely) - rl/mid-training with objective/data to make smaller CoT (most likely) - mid-training with more efficient arch: i would love for this to be true, and i can see how it's necessary if they use the previous base model generation and need efficient memory for long context, but since they also released some tricks with self-summary, i'd say unlikely? (they can and imo should be combined together for very long tasks)
Composer 2 is now available in Cursor.
3
5
2
134
28,548
Timothée Lesort retweeted
This week, we released a paper from Meta @AIatmeta “MuLoCo: Muon is a practical inner optimizer for DiLoCo”, showing that K=1 MuLoCo has a Pareto-optimal performance-training-time tradeoff Let’s drill deeper into single-worker MuLoCo’s efficiency 🧵1/5 xitter.gq/benjamintherien/status…
Are frontier LLMs trained across datacenters? One thing is certain: if the pre-training optimizer’s critical batch size is too small, they are NOT! Excited to announce MuLoCo, a pre-training optimizer that can efficiently pre-train across datacenters while having large enough batch sizes to warrant doing so. 🧵1/N
2
11
1
49
6,370
Timothée Lesort retweeted
I have open positions including Postdocs, PhD, master's students, and PhD interns. For more information eugenium.github.io/Projects/…
5
1
4
2,297
Timothée Lesort retweeted
Come see the Scaling Laws for DiLoCo poster at NeurIPS with @GabrielTeston and @NovaFallen8 !
3
8
54
4,186
Timothée Lesort retweeted
I mean it's a multimodal model (not inherently text-only), it's Instruct (not thinking), and it's compared with thinking models like K2 on non-thinking mode this is an uneducated take imo
It's kind of sad to see the slow death of Mistral. Their new model is: 1. Dumber than Deepseek 2. 3x more expensive than Deepseek 3. Slower than GPT-5
16
17
1
386
41,870
Timothée Lesort retweeted
Intensifying geopolitical competition leaves AI bridge powers in a difficult situation where they’ll soon likely face insurmountable barriers to independent frontier AI development. To stay relevant and thrive economically, they need to work together and strategically choose their AI development approaches.
The US controls 75% of global AI compute. China 15%, and the EU just 5% If bridge powers don’t react now, they’ll fall behind economically and militarily We need a plan Today, with 28 global experts (incl. @Yoshua_Bengio) we release a Blueprint for Multinational AI development
9
25
3
103
15,847
Timothée Lesort retweeted
🔥 We stress-tested today’s best AI code generators in 𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑐𝑦 ℎ𝑒𝑙𝑙. Introducing 𝐆𝐢𝐭𝐂𝐡𝐚𝐦𝐞𝐥𝐞𝐨𝐧 𝟐.𝟎: 328 challenges for version-controlled code generation. The verdict? Even top models only hit ~50% success.
3
25
1
44
5,466
Timothée Lesort retweeted
RAG and in-context learning are the go-to approaches for integrating new knowledge into LLMs, making inference very inefficient We propose instead 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗠𝗼𝗱𝘂𝗹𝗲𝘀 : lightweight LoRA modules trained offline that can match RAG performance without the drawbacks
1
15
2
46
8,802
Tired of tuning hyperparameters? Introducing PyLO! We’re bringing hyperparameter-free learned optimizers to PyTorch with drop in torch.optim support and faster step times thanks to our custom cuda kernels. Check out our code here: github.com/Belilovsky-Lab/py…
Have you ever trained a neural network using a learned optimizer instead of AdamW? Doubt it: you're probably coding in Pytorch! Excited to introduce PyLO: Towards Accessible Learned Optimizers in Pytorch! . Accepted at @icmlconf ICML 2025 CODEML workshop 🧵1/N
2
7
30
2,156
Is AdamW the best inner optimizer for DiLoCo? Does the inner optimizer affect the compressibility of the DiLoCo delta? Excited to introduce MuLoCo: Muon is a practical inner optimizer for DiLoCo! 🧵arxiv.org/abs/2505.23725 1/N
2
26
2
84
7,872
Timothée Lesort retweeted
MuLoCo: Muon x DiLoCo = ❤️ arxiv.org/abs/2505.23725 from @benjamintherien, Xiaolong Huang, @irinarish, @ebelilov * Using Muon as inner optimizer * Add quantization of the outer gradient to 2 bits (!) * Add error feedback
4
18
3
140
11,938
Timothée Lesort retweeted
Explore fundamental questions surrounding large language models by applying for a postdoctoral position with our Generalization Team at FAIR Paris. If you're attending ICLR, visit our booth to connect with my colleagues and learn more about this exciting opportunity.
📷 Hello Singapore! Meta is at #ICLR2025 EXPO 📷 Meta will be in Singapore this week for #ICLR25! Stop by our booth to chat with our team or learn more about our latest research. Things to know: 📷 Find us @ Booth #L03 (Rows 3-4, Columns L-M) in Hall 2. 📷 We're sharing 50+ publications & 15 workshops. That’s not all, we are actively hiring researchers to join us and help us build tomorrow’s reality: 📷 AI Research Assistant (PhD) (Paris) - meta.careers/3YLca3Q 📷 Postdoctoral Researcher (PhD) - Core Learning (Paris) - meta.careers/448CE2W 📷 Research Scientist, 3D Generative AI (PhD) - meta.careers/4cPhbOQ 📷 AI Research Scientist, Language - Generative AI - meta.careers/3GpoiRW #AI #PhD #Research #MetaCareers
4
19
1,407
Timothée Lesort retweeted
30+ accepted papers 6 oral papers 6 guest speakers join us at @iclr_conf on the 27th Hall 4 #3 for a full day of workshop on Modularity for Collaborative, Decentralized, and Continual Learning sites.google.com/corp/view/m… @derylucio, Fengyuan Liu, and myself will be organizing that day in person
Workshop alert 🚨 We'll host in ICLR 2025 a workshop on modularity, encompassing collaborative + decentralized + continual learning. Those topics are on the critical path to building better AIs. Interested? submit a paper and join us in Singapore! sites.google.com/corp/view/m…
3
26
10
102
61,782
How do MoE transformers, like DeepSeek, behave under distribution shifts? Do their routers collapse? Can they still match full re-training performance? Excited to present “Continual Pre-training of MoEs: How robust is your router?”!🧵arxiv.org/abs/2503.05029 1/N
1
21
1
39
3,778
Timothée Lesort retweeted
Great research on open-source by @Harvard: - $4.15B invested in open-source generates $8.8T of value for companies (aka $1 invested in open-source = $2,000 of value created) - Companies would need to spend 3.5 times more on software than they currently do if OSS did not exist I suspect that these numbers and impact are even greater for AI than for software (would be great to study!)
130
1,375
216
5,662
793,248
Timothée Lesort retweeted
Workshop alert 🚨 We'll host in ICLR 2025 a workshop on modularity, encompassing collaborative + decentralized + continual learning. Those topics are on the critical path to building better AIs. Interested? submit a paper and join us in Singapore! sites.google.com/corp/view/m…
4
37
12
137
101,199