Enterprise AI engineering & deployment @MistralAI | Views my own
New York
Joined August 2021
- Tweets6.3K
- Following2.4K
- Followers1.3K
- Likes40.6K
Howe Wang retweeted
Never felt closer to a fundamental change in how we build retrievers, and a different kind of memory for agents.
ARI is basically test-time training for retrievers, where gradients derived from a reasoning model's preferences teach the retriever where to look *at search time*.
Lots more to do here to get oblique search queries as manageable as non-oblique ones, and that will include first figuring out sub-linear (late-interaction-esque) search.
But the path ahead has never been clearer. Read @dianetc_'s blog!
Is there a way to scale a reasoning model’s judgment for searching a massive corpus?
We introduce the Amortized Reasoning Index (ARI), a new kind of retriever that updates its weights during search, using gradients derived at inference time from the ranking preferences of a reasoning model.
Every session, current agents spend lots of reasoning deciding which search results matter, and then throw all of that judgment away upon compaction. ARI is our attempt to “amortize” that judgment, so that reasoning spent on past queries keeps improving search on future ones.
With no relevance labels, ARI takes a 400M cross-encoder that starts out weak and grows its Recall@100 on held-out queries by up to 7×, ending above Qwen3-Embedding-8B on various oblique/reasoning-intensive search tasks.
Read our blogpost: dianetc.com/musings/ari/
Preliminary results and discussions are below!
Howe Wang retweeted
everyone is a data scientist now. learn to eval.
Howe Wang retweeted
100%. Enterprises were mostly unsuccessful building production quality classical machine learning models and modern AI model building is a lot harder to do well. You’d need to produce 10x cost or quality improvement to get people to bite and I don’t think that’s there in most cases.
Howe Wang retweeted
There are two main types of benchmarks:
1. those that measure frontier intelligence (terminalbench, gdpval, etc.). accuracy matters more than cost.
2. those that measure specialized intelligence. the pareto curve of cost/accuracy more than accuracy alone.
both are important, but they are distinct benchmark categories.
* as (1) saturates, they become (2).
* the largest frontier models at xhigh likely already saturate (2), but they do so at a cost/latency point that's completely price-prohibitive.
* every user should define (2) for the tasks they care about the most.
we're big fans of both. we have so far dedicated outsized attention to (2) especially related to document tasks.
we fundamentally think users should have the most price-efficient tools to understand, extract, and read over their documents. we've built parsebench, extractbench for this purpose.
we are starting to invest in (1) as well, because certain document tasks are so challenging that even frontier models struggle (e.g. see extractbench pro). we are continuing our investment in long-horizon workflow research at @llama_index
Howe Wang retweeted
Catching up on the 10 (!) open LLM architecture releases from the past month. Quite a lot happened in October.
I guess we can call it "Open October"!
1. Kolibri-1 (78B): A notable model from Germany. Features classic Grouped Query Attention (GQA) and Sliding Window Attention (SWA) attention module.
2. Ling 3.1 Flash (560B): Uses Kimi Delta Attention and the eighth uses gated Multi-head Latent Attention (MLA) inspired by DeepSeek.
3. Muse Spark Open (816B): Besides Musse Glimmer, the first major open weight release from Meta (since Llama). Uses classic GQA + SWA but adds a LatentMoE.
4. Nemotron 3.5 Super VL (120B): Combines Mamba-2 layers with just eight attention layers. Also LatentMoE
5. Dory1.0 (4B): A looped transformer design.
6. Reflection Beam (501B): GQA+SWA architecture.
7. Recursive Flint (40B): Note that "Recursive" is the company name; the architecture is a fairly standard MoE with GQA and SWA.
8. MiniMax M3.1-Flash (428B): Uses their own MiniMax Sparse Attention module.
9. LFM3 (1.7B): Hybrid architecture that mixes short convolutions and attention.
10. Mistral Large 4 (1T) combines MLA with sparse attention.
Will probably cover/discuss things in more depth in the coming weeks!
Howe Wang retweeted
Fireworks serves >40T tokens a day of "specialized models." I'd define this as any post-trained model for a customer.
Howe Wang retweeted
ML4 is now open-weight! Frontier capabilities on code agents & cyber, way more efficient and compact than its Preview version three weeks ago (!) and cherry on the cake: technical report telling its epic backstory. Check it out: huggingface.co/mistralai/Mis…
Howe Wang retweeted
OK, we drop it here: alphaxiv.org/abs/2611.mistra…. Comments welcome!
Howe Wang retweeted
We just released a technical report on the training of ML4
ML4 is now open-weight! Frontier capabilities on code agents & cyber, way more efficient and compact than its Preview version three weeks ago (!) and cherry on the cake: technical report telling its epic backstory. Check it out: huggingface.co/mistralai/Mis…
Howe Wang retweeted
Le Chonk (Mistral Large 4) technical report is out.
Worth a read. 👀
ML4 is now open-weight! Frontier capabilities on code agents & cyber, way more efficient and compact than its Preview version three weeks ago (!) and cherry on the cake: technical report telling its epic backstory. Check it out: huggingface.co/mistralai/Mis…