Cyrus Maher (@CyrusMaher) header
Cyrus Maher (@CyrusMaher) profile picture

Principal Scientist at Apple Health. Formerly @Vir_Biotech, @GrailBio and @humanlongevity. AI/Stats/ChemBio applied to cancer/aging/infectious disease.

San Francisco, CA
Joined August 2012
UMAP 0.6 is out! A new initialization, new optimizers and faster nearest neighbour search make UMAP much faster and far better at large scale. On 3 million arXiv papers it finishes in under 5 minutes, before UMAP 0.5 has even found its nearest neighbours.
24
110
18
604
26,697
Cyrus Maher retweeted
Introducing e-means, an algorithm beating k-means in 10x fewer epochs. With order of magnitude wall-clock speedups, e-means is released in FAISS and needs no hyperparameter tuning beyond k. It is robust to various scales and datasets. Still skeptical ? Just try e-means [1/10]๐Ÿงต
150
1,231
257
10,233
894,913
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context. Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7ร— faster and full attention 72.4x faster. With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
100
338
82
2,902
508,729
Cyrus Maher retweeted
Alright I finally have more to share on my personal project! Resin is a runtime for building fast LLM inference engines. The idea is to make all the generic parts of inference reusable, and leave only the model-specific forward pass + kernels to be written for each architecture / hardware target. To build an engine: 1. Import Resin's core. 2. Have an agent write a forward pass for the model youโ€™re interested in, for your specific hardware (GPU or TPU or AMD), any kernels you want. Then, boom, you have a standalone inference engine. โ€” I've been hacking over the past month (early post: xitter.gq/ekzhang1/status/208950โ€ฆ), and recently this week I got a lot of features + parallelisms working, inspired by DeepInfer. For DeepSeek-V4.1-Flash, it's nearly as fast as DeepInfer (within 5-10%), while being self-contained and supporting other models, plus metrics + HTTP server + prefix cache. Yesterday I wrote a skill for authoring models and put Codex to work on it overnight. The agent built a GLM-5.3-Flash engine with MTP, reaching up to 60,000 input tok/s and 16,000 output tok/s on B200 (EP4). So far inference "vibe coding" has not been very reliable or successful. I think it's because people start from scratch each time and reinvent lots of tricky things on the scheduler and server layers. But with a shared core, we can automate lots of inference work. The tricky bit is writing a fast forward pass. But Resin turns that into a standalone engine by just importing a library, and it becomes a verifiable goal for the AI. Code: github.com/ekzhang/resin
Soโ€ฆ I have it working now. The core of an inference engine in Rust is about 3k lines (scheduling, kv/swa/recurrent state, prefix cache, multimodal, dynamic batching) I then asked Codex to implement (with MTP, cudagraphs, TP on Blackwell): - DeepSeekV4 Flash - Gemma 3.8 27B - GPT-OSS - Inkling It turns out that most models take a lot of code to run with fast, persistent graphable kernels and so onโ€ฆ on the order of 2.5k for a simple model (gpt-oss) and up to 7k for a monster like DeepSeekV4 Flash that has mHC, CSA, HCA and DSpark Not sure if Iโ€™ll continue with this, but it was fun! The simple core is reusable, and startup times are crazy fast for an inference engine with no tech debt and not having to serve a ton of dynamic configurations - just one model forward pass in nvfp4 :โ€™) - maybe one day SGLang/vLLM will be modular like this. See interactivity-throughput below for Inkling-Small TP=1 after a few sconv kernel fusions. (This is without overlap scheduling btw! Donโ€™t even really need it when your core + scheduler is small and Rust.)
115
105
33
1,515
105,606
Cyrus Maher retweeted
Argon has this amazing natural understanding of the world. It can do so many things from first principles. Here's a pelican riding a bike, using fourier epicycles. Video made by Argon. Time-synced Lyria 3.5 music prompted by Argon.
235
591
240
8,422
2,250,745
Cyrus Maher retweeted
New Interpretability Research! ๐Ÿ†• We introduce the J++ Lens: an improvement to the J-Lens that reads out latent concepts more reliably and at earlier layers. We see 55% more faithful readouts than the J-Lens Core idea: Filter noisy gradients before averaging them together ๐Ÿงต1/11
9
47
4
398
15,588
Cyrus Maher retweeted
GLM-5.3-Flash on a single RTX PRO 6000 Blackwell 96GB: -67 tok/s decode (1 user), 110 tok/s total at 4 users - ~3,000 tok/s prefill - 200K context, needle passes at 194K 3.5ร— the 3090 offload recipe. Rebuilt @โ€‹0xSero's EXL3 offload image for sm_120 + retuned for 96GB ๐Ÿ”ฅ
15
1
3
100
21,115
Still running OCR over your PDFs before you can search them? Perplexity's new open embedding models are built to skip that step. Perplexity released pplx-embed-v2-late on Hugging Face under the MIT license, in 0.6B and 9B sizes built on Qwen3.5. They're ColBERT-style late-interaction retrievers: every token gets its own 128-dimensional vector, and a query is scored by matching each of its tokens to the closest one in the document (MaxSim). They handle text, images, and rendered pages such as PDFs and slides. Both sizes share one embedding space, so the small model can query an index built with the 9B. On the image track of the public ViDoRe v3 benchmark, Perplexity reports nDCG@10 of 62.3% for the 0.6B and 65.2% for the 9B. Both were distilled from an internal 18B teacher. Storing a vector for every token makes these indexes big, so test on your own data first. Index a few hundred of your pages with the 9B, compare results when you query with the 0.6B and the 9B, and note how much disk the index takes. ๐Ÿ” #RAG huggingface.co/perplexity-aiโ€ฆ
3
27
2
217
20,383
Cyrus Maher retweeted
Can โ€œchickenโ€ make a base model reason? ๐Ÿ” Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make โ€œchickenโ€ elicit reasoning too. blog: sophielwang.com/cues
49
82
29
626
200,579
Cyrus Maher retweeted
EmbeddingGemma 2 is out! Our first native multimodal embeddings built on Gemma 4 under Apache 2.0. ๐Ÿค—ย Embeds 100+ languages, code, images, audio, and video into one vector. ๐Ÿงฎย 8,192 context 3๏ธโƒฃย 740M omni, 440M text+vision, 570M text+audio, or 270M text-only ๐Ÿ“ˆย +14% jump on MTEB Code ๐Ÿช†ย Matryoshka: 768d, 512d, 256d, or 128d ๐Ÿค—ย Available in Sentence Transformers and LiteRT-LM ๐Ÿ”“ย Apache 2.0 huggingface.co/google/embeddโ€ฆ
22
60
4
574
35,519
Cyrus Maher retweeted
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
435
1,009
490
9,796
2,695,498
Cyrus Maher retweeted
I did promise you local model inside of ChatGPT! Point your LLM at this to have it configure itself One half of the repo is a proxy needed (could just be one host somewhere), the other half is the per-machine changes needed to your ChatGPT config github.com/muellerzr/local-cโ€ฆ
11
8
2
50
5,416
๐ŸšจTypical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them. We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute? โคต๏ธ Pedagogical RL
17
97
17
614
143,089
Cyrus Maher retweeted
SFT is not dead! ๐Ÿฅณ We found a way to make SFT rival current prevailing posttraining methods, often generalizing better and forgetting less than RL and OPSD. ๐Ÿคฏ Following our prior work on reasoning with sampling, we now introduce sampling to the posttraining stack. 1/n
77
245
66
2,166
370,487
Apple presents LoopCD: - Enables halving the number of recurrent loops while still matching or exceeding full-depth baselines - AIME 2024 pass@1 from 61.88% to 73.33% arxiv.org/abs/2610.02185
18
69
8
532
32,875
Cyrus Maher retweeted
The Cohere Embeddings team has been cooking ๐Ÿณ Several key innovations in the model: - The model not only retrieves relevant results, but finds the most human preferred documents based on new training & evaluation techniques. - Strong focus on multilinguality ๐Ÿ‡บ๐Ÿ‡ณ - just retrieving English docs is boring - Massive throughput gains - scale to any data size Together with a strong focus on privacy - your data stays private, always.
Introducing Cohere Embed 5: our new state-of-the-art family of embeddings models. Get frontier capabilities with Embed 5 Pro or low-latency performance with Embed 5 Fast.
5
19
1
102
13,356
Cyrus Maher retweeted
Preventing loss of control is an open scientific problem. Science requires sharing and debating evidence in public. We can't agree on safety standards unless companies and third-party evaluators publish far more concrete evidence about risk. planned-obsolescence.org/p/eโ€ฆ
11
67
8
419
46,321
Cyrus Maher retweeted
You can now run an LLM which scores higher than Sonnet-5 medium, and GPT-6-Sol medium for 3000$ in compute - 65 decode tok/s - 2000 prefill tok/s - 200k kv cache 1. RTX 3090 / 4090 / 5090 / Intel Arc B70 / AMD 2. 64 GB of RAM 3. 100 GB of NVMe Recipe today
GPU poor rejoice! You will be able to run Qwen3.8-Next-Flash on a single 24-32 GB GPU + 64GB ddr4/ddr5 + 100GB NVMe/SSD RTX 3090: - 2000 / 2800 tok/s prefill - 64 tok/s decode c1 - 210k fp8 cache Intel Arc B70: - 800 / 1200 prefill tok/s - 35 decode tok/s - 270k fp8 cache
66
38
10
1,230
145,601
Cyrus Maher retweeted
Most open source RL environments tend to live on two extremes of a spectrum: - Simple, educational games like Wordle to learn how RL works - Complex, long-horizon tasks like Terminal Bench Science that are designed to probe the limits of frontier models We found that there's a huge gap in the middle, especially for those wanting to improve the coding capabilities of models <10B parameters. Today we're filling that gap by releasing >5k RL environments that are specifically designed for real-world data science tasks and are a great testbed for developing the next variant of GRPO or OPSD ;) Awesome release by Adithya who single-handedly built the envs, defeated dozens of challenges with async RL and figured out how to make all this run on a single GPU ๐Ÿ”ฅ
Releasing SmolDataEnvs ๐Ÿค— 5K+ Verifiable RL Environment tasks for hill-climbing small models in code and data science. Completely open source: environments, evals, training
7
21
2
253
20,683
Cyrus Maher retweeted
Speech recognition is easyโ€”until you ask it to listen forever. Today weโ€™re open-sourcing Audio8 ASR Infinite: Ultra-low latency, unlimited audio, 24/7 transcription, no drift. Built-in semantic turn detection keeps it listening like a human ear. New SOTA for streaming ASR.
52
198
25
2,315
167,521