LLMs @ FAIR, prev. DINO stuff @ INRIA & FAIR
Joined March 2021
- Tweets1.6K
- Following856
- Followers4.9K
- Likes5.2K
TimDarcet retweeted
Can LLMs draw objects they never saw?
I asked 5 text-only LLMs (trained without vision) to draw simple things. In text. No tools allowed.
Interesting results
TimDarcet retweeted
i own a relatively new (2025) samsung tv.
i kept getting weirdly targeted ads.
i tried reversing the firmware but it was aes encrypted and no keys unless tear it apart and do uart/jtag magic.
zero days it is. root shell on new samsung TVs!
discovered PRIVACY HORROR. write up?
TimDarcet retweeted
Looks like we have over 500 submissions to ICLR this year.
openreview.net/group?id=ICLR…
TimDarcet retweeted
Today we published a technical report on Muse Glimmer, our 30B open-weight model built for agentic tasks on consumer hardware.
It runs at practical speeds on a Mac or PC with a single consumer GPU at 4-bit precision, and leads comparably-sized open models on tool use and multi-step planning.
Read about the model architecture, distillation recipe, quantization and more: go.meta.me/4Bcvxol
TimDarcet retweeted
One day we'll tell stories about how people thought language models were intelligent with the same incredulity of people thinking that cameras steal your soul, or that the train in the movie was actually going to crash into people.
TimDarcet retweeted
CLAUDE FOUND THE SEEDS
anthropic.com/research/ecdsa…
TimDarcet retweeted
Congrats to Meta on building the best open-weight model in the United States !
Today we’re releasing Muse Spark Open, the most capable US open-weight model to date.
Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning.
Key Model Specs:
→ 816B total / 42B active parameters
→ 1M token context window
→ Natively multimodal across text, image, video, audio
TimDarcet retweeted
1/ muse spark open is the most capable US open-weight model to date.
816B total params, 42B active. 1M-token context. natively multimodal. built for coding and agentic work like long-horizon tasks, hard software engineering, multi-step reasoning.
america! 🇺🇲
TimDarcet retweeted
so how long before we effectively apply this to bridge the communication barrier with intelligent animals?
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair.
It even works when the images and the captions come from different datasets.
Project page: dominik-schnaus.github.io/un…
⬇️
TimDarcet retweeted
This is a result I’ve dreamt about for many years: *unpaired translation between images and text*
I thought it might be only slightly possible, the kind of thing you have to really squint at. But Dominik proved this wrong. You don’t have to squint. Worth looking for yourself:
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair.
It even works when the images and the captions come from different datasets.
Project page: dominik-schnaus.github.io/un…
⬇️
TimDarcet retweeted
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair.
It even works when the images and the captions come from different datasets.
Project page: dominik-schnaus.github.io/un…
⬇️
TimDarcet retweeted
I'm starting to think that most AI coding slop comes from people being too complacent with their agents.
They just trust what the agent outputs without challenging it.
I am giving a hard time to the coding agents I use. Not sure any human would accept what I do.
It is not about being impolite (I am polite). It is about being extremely demanding.
TimDarcet retweeted
Today, we are launching a preview of our new model, Mistral Large 4 (ML4), aka le Chonk 🐈.
ML4 is a 1T-parameter model with 49B active parameters, trained natively with multimodal capabilities.
It is at the frontier of open models, and by far the strongest open-weight model from the US or Europe.
The RL run behind this preview is still in flight and shows no sign of saturation -- we will release a final version before the end of the month along with the weights of the model.
🧵
1/n
Meet Mistral Large 4, aka Le Chonk.
• 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
• State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding.
• Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure.
• Available to all via API today. Working with cybersecurity partners privately.
Open weights release end of October.
TimDarcet retweeted
when you have crazily large max logits, FA kernel is fragile.
(large mean key is kinda the same as large max logits, cuz mean key can shift logits)
but it's nonsense that model needs logits larger than 100.
exp(100) is more than 10^43.
I think it's an optimization problem.
We trained an LLM with FlashAttention-3 in BF16 🔥 Training looked healthy for a long time. Then the grad norm shot up 1000× 📈 and the loss ended 0.2 nats above FP32 attention. Not a single NaN 😶
The cause is in the attention backward 🧵
📄 arxiv.org/abs/2609.34272
