software engineer in NYC. prev https://xitter.gq/t.co/Bx70AilrPF and https://xitter.gq/t.co/ZTUpgwSh6a. Now @powersetres.
New York
Joined November 2007
- Tweets997
- Following1.2K
- Followers873
- Likes11.7K
I think using embeddings with “traditional ML” is really cool and maybe under explored (or at least under hyped) for unlocking a different and cheaper kind of intelligence
EmbeddingGemma 2 maps text, images, video, audio, and code, running 100% locally on your device.
At just 740M parameters, you can build powerful multimodal apps running entirely in your browser via WebGPU & Transformers.js. From multimodal search to cross-modal classification and clustering, local embeddings unlock endless possibilities.
not just semantic search or clustering, but for example regression: arxiv.org/abs/2411.14708
Patrick DeVivo retweeted
It's been a while since I showed any Jev stuff but I've been cooking on a way to use @GoogleDeepMind's EmbeddingGemma2 as an insanely fast real-time /v1/systemone endpoint (20ms on text, 100ms on images).
It's trainable in real-time, can distill an upstream teacher model, and is multi-modal by default. Hard to believe a @googlegemma model of this size can do so well with the right harness!
on @PowersetRes we run an LLM embedding pipeline to categorize repos based on their READMEs and website content.
we use embedding similarity to a category centroid to determine membership- it works, but seems like a place Jev could really shine
could simplify a whole lot and probably be faster/cheaper
Patrick DeVivo retweeted
12,000 tok / sec on a $250 board is nuts
Waiting for LLMs to stream text is going to seem so quaint in a few years...
12,000 tok/s on a 250$ FPGA board. It's a small model, and doesn't answer coherently to any question or prompt really. try it out on trylittlelamb.com/ but don't expect a good chat. at least its VERY fast
I remember interviewing back in 2015 thinking it would be edgy and a way to differentiate myself to answer a question: "how would you implement a photo gallery feature?" truthfully by saying I would search stackoverflow for the most suggested "photo gallery" library and just use that...
didn't get the job
I'm surprised there's not more hype for @GoogleDeepMind's Gemma 4
It absolutely flies on @cerebras and has been just as good for certain research tasks (and orders of magnitude faster and cheaper) than OpenAI models we were using
switched from Claude Code to Codex CLI today and am experiencing this in real time. I can’t articulate exactly why…it’s just better
Less sycophantic? Feels more comprehensive? 5.5 seems to just miss fewer things
Patrick DeVivo retweeted
The best recruiting signal is often sitting in public GitHub data.
Today, we’re open sourcing the data layer we use to help founders recruit exceptional engineers.
research.powerset.co/posts/r…
the job of software engineer is so clearly no longer about directly managing code, but managing the systems and rails for AI tools to ship the best possible codel, given the domain and its constraints