Engineer at @turbopuffer, Lucene committer
Caen, France
Joined April 2009
- Tweets2.7K
- Following649
- Followers2.5K
- Likes8.9K
I wrote this post a few months ago but held off on sharing it because it takes a lot of shortcuts. Today, colleagues asked me why Lucene uses segments, I walked them through it, and it clicked. So here it is :) jpountz.github.io/2026/04/19…
Adrien Grand retweeted
Working on that blog post as fast as I can ;). Excited about this one. Dynamic rescoring, no wasted compute by static oversampling, spending compute for recall only when necessary.
One thing I've come to appreciate while working on v3 (turbopuffer.com/v3) is how much easier it is to change the underlying storage/query engine with a pure SaaS product.
Two new implementations of the vectorized variant of block-max MAXSCORE that I introduced in Lucene in 2025 were just released: in ParadeDB (github.com/paradedb/tantivy/…) and Luxir (github.com/luxir-search/luxi…).
It's slowly replacing block max WAND as the go-to algorithm for text search.
Adrien Grand retweeted
we're changing tpuf's storage architecture so we can keep pushing the search frontier
the new engine (v3) now passes CI, but we have to make it fast before we ship it to prod
we're grinding query plans in public until v3 wins on everything
follow along: turbopuffer.com/v3
Adrien Grand retweeted
I am glad to see seminal & long standing projects like Apache Lucene adopting sane & simple AI policies for the community. Use LLMs for code, not communication. We are all tired of talking to an LLM with just somebody copy-pasting in the middle.
Adrien Grand retweeted
RLucene is public now! 🎉
Feel free to jump in and help make this Rust version of Apache Lucene better!
github.com/Rustify-All/rluce…
Adrien Grand retweeted
If a library has obvious bugs I don't "prompt" the maintainer to fix it. It has to go.
OTOH I have read more books than code but I the impact of reading code has been bigger. I spent 4 hours on how Lucene stores data and I have used that in several places. I should do it more.
I never considered the challenge of replicating Git repositories before, it's fascinating how similar it is to Lucene indexes:
- NFS: doesn't work
- storing data in Cassandra (Lucandra): too slow to be practical
- application-level replication of operations or segments: works
We're making Git hosting more reliable, performant, and scalable.
This post traces 20 years of Git infrastructure and explains how that history led us to design and operate our Git storage, Origin, as if it were a database.
cursor.com/blog/git-at-any-s…
Adrien Grand retweeted
I did a write-up of things I learned during my 10-year tenure at GitHub, the things that worked, and what we're trying to improve in Origin.
The scale of the challenge is staggering, but I really want people to know we're putting a lot of thought and care into it.
We're making Git hosting more reliable, performant, and scalable.
This post traces 20 years of Git infrastructure and explains how that history led us to design and operate our Git storage, Origin, as if it were a database.
cursor.com/blog/git-at-any-s…
Adrien Grand retweeted
Today we're announcing our $12.5M seed, led by B Capital and Speedinvest.
In 8 months, a small team built a news index that's faster and more accurate than any search API we tested, and that hands agents full, fresh documents instead of snippets.
Adrien Grand retweeted
Creator of Sqlite on pull requests: "You say, oh, it's free. No. It's not free. What you're doing is asking me ... to maintain it for you, to to document it for you, to test it for you, to maintain it for you for the next 25 years. That's not free." Yep.
Wise words from a wiser man than me. I've told people for the past decade and I have recent posts on here saying the same: the merge button is the easy part. Its the decade+ (Richard says 25 years) that follows where you've accepted the transfer of maintenance thats hard.
To keep my blog alive, I wrote a short blog on a lesser-known aspect of Lucene that I find quite cool: how it tries to create self-optimizing query plans instead of choosing between multiple plans. jpountz.github.io/2026/06/04…
Adrien Grand retweeted
Does a lexical retriever suffice for agentic search when agents can keep refining their queries?
As LLMs become more capable in agentic loops, agents can continuously refine their actions based on environmental feedback. We couldn’t help but ask the question above.
Adrien Grand retweeted
BM25 efficiently scores text, but relevance often depends on more than text (recency, popularity, PageRank)
we score non-text attributes as clauses in the same MAXSCORE plan that evaluates BM25 → better first-stage relevance, still scales to 100M+
tpuf.link/rank-by-attr
Adrien Grand retweeted
Full-text search was solved in 2016, Lucene had owned the space for a decade. Rewriting it in Rust, as a side project, on a plane sounds like a terrible idea.
But Paul Masurel (@fulmicoton) is a special kind of person. His side project became Tantivy, now a widely used (and very competitive) search library that's even helping improve Lucene.
We sat down and talked about challenging solved domains, open-source competition done right, and why long-fermented frustration is an underrated driver.
paradedb.com/blog/tantivy-in…
I was curious how SereneDB manages to perform well, on long queries in particular. Answer: it implemented the same block-at-a-time vectorized MAXSCORE algorithm as Lucene: github.com/serenedb/serenedb…! To my knowledge, this is the third implementation of this algorithm, very exciting.
Adrien Grand retweeted
Here comes a new challenger.
iresearch an engine implemented by SereneDB enters the contest with a bang!
tantivy-search.github.io/ben…
Adrien Grand retweeted
Apache Lucene had a pretty awesome 2025. While I am getting slower as I get older. Lucene is getting faster. elastic.co/search-labs/blog/…
Adrien Grand retweeted
btw hiring for a systems eng. role at @turbopuffer soon on the text team, you’d work closely w/ @jpountz and I on billion-scale {storage,ranking,query eval}. DMs open!