I'm excited to announce Pretense, a library for pretraining sentence transformers! It supports a variety of techniques such as contrastive learning, RetroMAE, coCondenser, and more!
uv add pretense / pip install pretense, and you're good to go! v0.1.1 is out.
Details in 🧵
gautham retweeted
At “Women in Systems”, we will feature a panel discussion on the evolution and broader development of systems with Sujatha Sivaramakrishnan, Meera Sundar, and Janhavi Tripurwar at Microsoft.
Registrations close on Tuesday, the 24th.
See you there! hasgeek.com/bengalurusystems…
Super nostalgic models
For all my Sparse embedding model enthusiasts, the ever-excellent GTE-MLM line is now natively integrated in Transformers!
The original repositories still require trust_remote_code=True and don't work with the latest transformers anymore though...
So I made these native copies
7
199
One of the smartest and kindest engineers I've ever met. He brings out the best in everyone, any org would be lucky to have him
I recently got laid off from Couchbase and I'm looking for new roles right now - I bring experience working on high-density production storage systems (LSM Trees) and I'm really curious + actively learning about Inference and AI infra in general. I also keep up with research 1/n
8
154
lol
Embeddings vs. Grep: Rethinking the Dense Advantage in Code Search
Ant Group finds that dense retrieval's lead over BM25 in code search shrinks under fairer evaluation.
📝 arxiv.org/abs/2611.20455
👨🏽💻 github.com/hq-bench/repo-pre…
1
2
283
never a dull day in the singularity
NVIDIA announces CEO transition. Read the release: nvidianews.nvidia.com/news/n…
2
61
Lol @tomaarsen we got an embedding model from meta after all
Meta just released MetaEncoder on Hugging Face
A unified multimodal encoder for text, images and video, with SOTA retrieval and decision support.
1
5
336
I haven't even gotten around to the e-means paper
Bro is really shipping!
Together with e-means, we are officially releasing the first 10 billion scale benchmark for clustering and search : Dino10B.
It consists of 10 billion embeddings from Yahoo Flickr image patches computed with a Dino ViT-L/16 model.
Blog post at : swannbessa.github.io/blog/di…
1
4
333
gautham retweeted
We're super excited to announce that we're offering travel grants to support women attending "Women in Systems" in Bengaluru on the 28th, so feel free to fill in the details in the form below if you're a traveller!
docs.google.com/forms/d/e/1F…
gautham retweeted
We are thrilled to host a special meetup, "Women in Systems", at Microsoft (Bellandur) for our November edition on the 28th, featuring lightning talks as well as a panel discussion.
This meetup is open to everyone, so feel free to register!
hasgeek.com/bengalurusystems…
Lots of interesting chip developments over the past few weeks. AMD winning, pytorch getting actual proper support for TPU and neuron (rocm when?). Nice to see actual competition
ALERT🚨🚨: AMD MI355 is 1.42x FASTER THAN GB300 NVL72. THIS IS ACHIEVED USING THE TILERT SOFTWARE, WHICH GETS 1330 p90 tok/s/user on long context workloads!
Thanks to AMD leadership for providing an MI350 cluster to TileRT maintainers, enabling them to achieve this great result!
4
146
gautham retweeted
That's a wrap for the October Edition, this time with @lossfunk!
Here's what went down:
- @kaxsthxbh walked us through how inference works from first principles.
- @DivyanshGolyan gave us a look inside Rui, a Zig agent harness.
(1/2)
I wonder if you can extend this to embedding models by using the top K ranked judgments for an ordinary embedding model as well. Basically train the cross-encoder + an embedding model!
Is there a way to scale a reasoning model’s judgment for searching a massive corpus?
We introduce the Amortized Reasoning Index (ARI), a new kind of retriever that updates its weights during search, using gradients derived at inference time from the ranking preferences of a reasoning model.
Every session, current agents spend lots of reasoning deciding which search results matter, and then throw all of that judgment away upon compaction. ARI is our attempt to “amortize” that judgment, so that reasoning spent on past queries keeps improving search on future ones.
With no relevance labels, ARI takes a 400M cross-encoder that starts out weak and grows its Recall@100 on held-out queries by up to 7×, ending above Qwen3-Embedding-8B on various oblique/reasoning-intensive search tasks.
Read our blogpost: dianetc.com/musings/ari/
Preliminary results and discussions are below!
1
6
266
The vector index-ifixation of attention continues! Very cool work
Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention.
ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵
1
5
292
Great release! But running a 105B moe model with your own super optimized inference, vs other way smaller models which probably don't have that same level of optimization, is a little disingenuous
Today we’re releasing Sarvam Prism, a System One model built on Sarvam 105B for fast, accurate decisions across enterprise workflows.
It scores 71.8 with a median response time of 44 milliseconds, the highest average score among decision models that respond in under 100 milliseconds.
1
7
511
Yikes man
Studying "emergent behaviour" might just be studying what tasks are deemed important enough to RL by labs
i replicated this and tried a subset of @rikiyatakehi et al's exact experiments on new and old models
its pretty clear that only the latest frontier models have these capabilities
7
410
someone needs to try this with SOTA open models, which are (probably) not RL'd for mechinterp
i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models.
2
3
429





























