Monomorphism retweeted
🔥 Serving coding agents well needs a high-performance prefill engine, but existing systems fall short.
🚀 Introducing SSEngine-GLM5.3-Flash: an open-source prefill engine for GLM-5.3-Flash on four B200 GPUs. It combines pipeline parallelism, a tiered prefix cache across GPU memory, DRAM and SSD, and a shortest-remaining-prefill-first scheduler.
📈 On real hardware, it sustains 3.1× (AgentX) and 3.2× (TraceLab) the request rate of any vLLM or SGLang setup we tested.
🧵 More in the thread below (1/8)
Monomorphism retweeted
🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token.
Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work.
📈 Across Qwen, Kimi, and GLM models:
• Matches FP32 accuracy
• 4.25× faster linear attention kernels
• Up to 2× higher inference throughput
• Cuts memory by up to 56%








