programmer, optimist. work: @turbopuffer
nyc
Joined November 2019
- Tweets1.6K
- Following291
- Followers2.1K
- Likes4.6K
plain ol' multi-lingual full-text search, still as relevant as ever!
new: Chinese, Japanese, and Korean tokenizers
smartcn, kuromoji, and nori split CJK text into known dictionary words, like 服装 (clothing). kuromoji and nori also match across word forms, so a search for 食べる (eat) finds 食べました (ate)
docs: turbopuffer.com/docs/fts#tok…
it’s exceptionally not cool to find someone’s personal phone number online and cold call them trying to sell to the company where they work at
two thoughts:
- will make damn well sure tpuf never uses pump.co
- objectively funny company name
latency(tpuf search w/ native embeddings) < latency(fetch embedding) + latency(tpuf search)
makes for some beautiful traces
now in beta: native embeddings in tpuf
embedding is the most painful part of puffing. we want to make it easy
you can now convert chunks to vectors as you read and write to turbopuffer, without extra calls to an embedding model provider API
docs: turbopuffer.com/docs/embeddi…
TIL: script.google.com lets you run automations on your calendar(s). in this case, running an hourly sync of a personal calendar (only specific events) to your work calendar as "busy" events
excited to get this out there in the hopes that other folks can benefit from our work here! a lot goes into making a good tokenizer
also wanna make sure I thank the tantivy folks, we've built our earlier tokenizers (everything before word_v4) atop their lovely open sourced work
so @morgallant has optimized FTS tokenization throughput to 423 MiB/s and open-sourced it (github.com/turbopuffer/alyze)
I keep telling him that it would be really high agency to get to DRAM bandwidth (~100 GiB/s), and he keeps getting annoyed at me and making it faster
fangirling over being on dash 8 for the first time in a while — i think my first ever flight was on a dash 8!!!
hoping to write a blog post on this eventually, it's a fascinating topic.. so many different possible approaches, each with their own tradeoffs. for v1, as is always the case at tpuf, we've optimized for simplicity
new in tpuf: regex indexes
regex and glob filters can now use a trigram index to avoid full-table scans
ALT A chart showing p90 query latency performance gains for regex and glob filters after the introduction of a trigram regex index on turbopuffer. Latency improved from 115 milliseconds to 33.8 milliseconds for a representative regex filter and from 267 milliseconds to 34.2 milliseconds for a representative globbing filter. The chart suggests that while these kinds of filters already worked, now they puff!
btw hiring for a systems eng. role at @turbopuffer soon on the text team, you’d work closely w/ @jpountz and I on billion-scale {storage,ranking,query eval}. DMs open!
joint work with @jpountz, covers some of the storage work behind the ftsv2 launch!
for FTS v2, we redesigned our inverted index structure
• tighter compression
• less KV overhead
• better MAXSCORE interaction
up to 10x smaller indexes → up to 20x faster text search!
tpuf.link/fts-index
1
6
2,255
still in awe of the talent density on the @turbopuffer team… truly such an honour and a privilege to get to work with these folks every day
I really hope that the continued success of Zig will help convince more systems programmers that treating memory allocations as infallible is a terrible idea for PLs, and that OS-level memory overcommits in Linux server environments is similarly terrible!
joint work with @jpountz and @nikhilbenesch; excited to start rolling this out more broadly! insane amount of progress on FTS recently :)
FTS v2: up to 20x faster full-text search
turbopuffer is now on par with Tantivy and Lucene for many queries, more to come
v2 now in beta. 2 PRs away from all query plans being implemented. will be enabled in prod for all, shortly.
ALT Horizontal bar chart comparing turbopuffer FTS v1 vs v2 latencies for five queries on English Wikipedia, v2 much faster (3–20ms) vs v1 (8–174ms).
hand-rolled a fully uax#29-compliant tokenizer today, brain is a little fried but it was lowkey pretty fun
one of my favourite ways to deploy to @Railway! build the binary locally, package it up in a light alpine image and ship it off
starting to learn a bit more about unicode and its intraticies. wrote a zig program to parse out property tables.. tried to do it at comptime, succeeded, but unfortunately made compilation times unreasonable. so instead, had to embed .txt in binary and parse it on the fly (sigh)