gautham (@capemox) header
gautham (@capemox) profile picture

angle born in a convex hull

Bengaluru, India
Joined July 2023
I'm excited to announce Pretense, a library for pretraining sentence transformers! It supports a variety of techniques such as contrastive learning, RetroMAE, coCondenser, and more! uv add pretense / pip install pretense, and you're good to go! v0.1.1 is out. Details in 🧵
3
3
31
1,472
At “Women in Systems”, we will feature a panel discussion on the evolution and broader development of systems with Sujatha Sivaramakrishnan, Meera Sundar, and Janhavi Tripurwar at Microsoft. Registrations close on Tuesday, the 24th. See you there! hasgeek.com/bengalurusystems…
7
20
893
idk if it's some form of confirmation bias but cracked search/IR folks seem to be always be from France or Italy
1
9
384
Super nostalgic models
For all my Sparse embedding model enthusiasts, the ever-excellent GTE-MLM line is now natively integrated in Transformers! The original repositories still require trust_remote_code=True and don't work with the latest transformers anymore though... So I made these native copies
7
199
One of the smartest and kindest engineers I've ever met. He brings out the best in everyone, any org would be lucky to have him
I recently got laid off from Couchbase and I'm looking for new roles right now - I bring experience working on high-density production storage systems (LSM Trees) and I'm really curious + actively learning about Inference and AI infra in general. I also keep up with research 1/n
8
154
Lol @tomaarsen we got an embedding model from meta after all
Meta just released MetaEncoder on Hugging Face A unified multimodal encoder for text, images and video, with SOTA retrieval and decision support.
1
5
336
I haven't even gotten around to the e-means paper Bro is really shipping!
Together with e-means, we are officially releasing the first 10 billion scale benchmark for clustering and search : Dino10B. It consists of 10 billion embeddings from Yahoo Flickr image patches computed with a Dino ViT-L/16 model. Blog post at : swannbessa.github.io/blog/di…
1
4
333
We are thrilled to host a special meetup, "Women in Systems", at Microsoft (Bellandur) for our November edition on the 28th, featuring lightning talks as well as a panel discussion. This meetup is open to everyone, so feel free to register! hasgeek.com/bengalurusystems…
11
2
43
2,372
Lots of interesting chip developments over the past few weeks. AMD winning, pytorch getting actual proper support for TPU and neuron (rocm when?). Nice to see actual competition
ALERT🚨🚨: AMD MI355 is 1.42x FASTER THAN GB300 NVL72. THIS IS ACHIEVED USING THE TILERT SOFTWARE, WHICH GETS 1330 p90 tok/s/user on long context workloads! Thanks to AMD leadership for providing an MI350 cluster to TileRT maintainers, enabling them to achieve this great result!
4
146
Imagine how I felt when I found out that the CTO of anthropic is an alum of my uni
can now say my college has an alum at anthropic! wild.
1
3
202
That's a wrap for the October Edition, this time with @lossfunk! Here's what went down: - @kaxsthxbh walked us through how inference works from first principles. - @DivyanshGolyan gave us a look inside Rui, a Zig agent harness. (1/2)
1
7
1
49
2,122
I wonder if you can extend this to embedding models by using the top K ranked judgments for an ordinary embedding model as well. Basically train the cross-encoder + an embedding model!
Is there a way to scale a reasoning model’s judgment for searching a massive corpus? We introduce the Amortized Reasoning Index (ARI), a new kind of retriever that updates its weights during search, using gradients derived at inference time from the ranking preferences of a reasoning model. Every session, current agents spend lots of reasoning deciding which search results matter, and then throw all of that judgment away upon compaction. ARI is our attempt to “amortize” that judgment, so that reasoning spent on past queries keeps improving search on future ones. With no relevance labels, ARI takes a 400M cross-encoder that starts out weak and grows its Recall@100 on held-out queries by up to 7×, ending above Qwen3-Embedding-8B on various oblique/reasoning-intensive search tasks. Read our blogpost: dianetc.com/musings/ari/ Preliminary results and discussions are below!
1
6
266
Reminder from todays AI Powered Search lesson, BM25 is vector search and always has been :)
1
16
459
Do people actually use the html viewer on arxiv? Doesn't look great on the eyes ngl
4
3
283
The vector index-ifixation of attention continues! Very cool work
Today we’re introducing Zyphra Hierarchical Attention (ZHA), a path to true subquadratic attention. ZHA searches a learned tree of compressed summaries, with projected speedups of 35× over dense attention and 5× over MSA at 10M tokens, increasing to ~400× and ~55× at 1B. 🧵
1
5
292
Great release! But running a 105B moe model with your own super optimized inference, vs other way smaller models which probably don't have that same level of optimization, is a little disingenuous
Today we’re releasing Sarvam Prism, a System One model built on Sarvam 105B for fast, accurate decisions across enterprise workflows. It scores 71.8 with a median response time of 44 milliseconds, the highest average score among decision models that respond in under 100 milliseconds.
1
7
511
Yikes man Studying "emergent behaviour" might just be studying what tasks are deemed important enough to RL by labs
i replicated this and tried a subset of @rikiyatakehi et al's exact experiments on new and old models its pretty clear that only the latest frontier models have these capabilities
7
410
someone needs to try this with SOTA open models, which are (probably) not RL'd for mechinterp
i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models.
2
3
429