Krish Agarwal (@wonderingkrish) profile picture

ECE PhD student @CarnegieMellon

Pittsburgh, PA
Joined April 2016
Krish Agarwal retweeted
Replying to @bespokelabsai
Congrats on the release of Multi-Agent Coordination Benchmark @bespokelabsai! ๐Ÿ”ฅ We actually came across the same observations in our recent Singularity-Bench (Released on Monday)! In a task to design an efficient large-scale serving system, Opus 5.5 actually managed to improve its score when required to use subagents, but GPT 6.1 Sol actually degraded its score when required to use subagents.๐Ÿคฏ In our case, solving the task effectively requires the agent to completely revamp the starter implementation, but when required to use subagents, GPT 6.1 Sol chose to just use separate agents to make fine-grained edits (local optimization) to the starter code, suggesting that it has a harder time with multi-agent coordination! ๐Ÿš€ Check out our work here: xitter.gq/InfiniAILab/status/212โ€ฆ
Introducing Singularity-Bench ๐Ÿ•ณ๏ธ ๐Ÿ”„ We measure whether agents can autonomously conduct the full agent development lifecycle: six long-horizon, end-to-end tasks spanning pre-, mid-, and post-training, inference engines, cluster management, and harness development on production GB300 clusters! We emphasize rigorous verification to guarantee correctness and execution integrity ๐Ÿ‘ฎโ€โ™‚๏ธ ๐Ÿ™Œ Big shoutout to all authors contributing from academia and industry: @IronSteveZhou, @wonderingkrish, @chenzhuoming911, Zerui Wang, @KyriectionZhang, Wenshuang Li, Xinpeng Wei, Zhichen Zeng, @Jingyu227, @JueWANG26088228, @Wu_Sen, Ziheng Jiang, @zhendongucb, @hancai_hm, @Hades317, @ying11231, @ce_zhang, @uiuc_aisecure, and @BeidiChen! How far are current agents from building their successors end to end? ๐Ÿค” We're here to find out! ๐Ÿงต1/n ๐ŸŒ Website: infini-ai-lab.github.io/singโ€ฆ
1
2
6
199
Excited to release Singularity-Bench! Opus 5.5 might not be able to autonomously develop its successor just yet, but it seems to be getting close ๐Ÿ‘€
Introducing Singularity-Bench ๐Ÿ•ณ๏ธ ๐Ÿ”„ We measure whether agents can autonomously conduct the full agent development lifecycle: six long-horizon, end-to-end tasks spanning pre-, mid-, and post-training, inference engines, cluster management, and harness development on production GB300 clusters! We emphasize rigorous verification to guarantee correctness and execution integrity ๐Ÿ‘ฎโ€โ™‚๏ธ ๐Ÿ™Œ Big shoutout to all authors contributing from academia and industry: @IronSteveZhou, @wonderingkrish, @chenzhuoming911, Zerui Wang, @KyriectionZhang, Wenshuang Li, Xinpeng Wei, Zhichen Zeng, @Jingyu227, @JueWANG26088228, @Wu_Sen, Ziheng Jiang, @zhendongucb, @hancai_hm, @Hades317, @ying11231, @ce_zhang, @uiuc_aisecure, and @BeidiChen! How far are current agents from building their successors end to end? ๐Ÿค” We're here to find out! ๐Ÿงต1/n ๐ŸŒ Website: infini-ai-lab.github.io/singโ€ฆ
4
1
9
433
Krish Agarwal retweeted
๐Ÿง  How can we keep scaling LLMs with minimal changes in their serving energy? DeepSeekโ€™s Engram and our concurrent work STEM showed the promise of static sparse memory: large embedding tables with efficient access. But can mature pretrained models learn to use a newly added memory within a limited training budget, avoiding costly pretraining from scratch? ๐Ÿš€ Introducing STAPLE: a recipe to extend pretrained models' abilities with static sparse memory. Serving energy per token stays close to dense and >50% below MoE upcycling in high-throughput regime. Scalable capacity, interpretable memory, and compatibility with MoE upcycling - without any meaningful increase in serving cost. ๐Ÿงต Paper: arxiv.org/abs/2610.12599 Code: github.com/Infini-AI-Lab/STAโ€ฆ Website: infini-ai-lab.github.io/STAPโ€ฆ
3
18
5
51
5,802
Krish Agarwal retweeted
How well can agents continually self-improve from serving experience? Weโ€™re excited to introduce our new benchmark, ๐’๐ž๐ซ๐ฏ๐ž๐‹๐ž๐š๐ซ๐ง๐๐ž๐ง๐œ๐ก: Real-world deployment is not a one-time test of capability. The knowledge agents need is often hidden and can change over time. ServeLearnBench tests whether agents can infer, apply, and revise this knowledge from interaction. Across 5 learning harnesses ร— 6 models, we find: 1. ๐‹๐š๐ซ๐ ๐ž ๐ฅ๐ž๐š๐ซ๐ง๐ข๐ง๐  ๐ ๐š๐ฉ๐ฌ ๐ซ๐ž๐ฆ๐š๐ข๐ง. Agents can solve tasks when the hidden policy is given, but struggle to discover it from experience. 2. ๐๐จ ๐Ÿ๐ซ๐ž๐ž ๐ฅ๐ฎ๐ง๐œ๐ก ๐Ÿ๐จ๐ซ ๐š๐๐š๐ฉ๐ญ๐š๐ญ๐ข๐จ๐ง. Continual learning can be costlyโ€”and can even degrade already-correct behavior. 3. ๐„๐ฑ๐ฉ๐ฅ๐จ๐ซ๐š๐ญ๐ข๐จ๐ง ๐ข๐ฌ ๐š ๐ค๐ž๐ฒ ๐›๐จ๐ญ๐ญ๐ฅ๐ž๐ง๐ž๐œ๐ค. Better exploration strongly correlates with better adaptation.
2
8
4
43
11,209
Happy to share our recent work on FlashRT โšก Multimodal applications are highly diverse. Most systems work still falls on human developers, which is unscalable. FlashRT automates this by guiding coding agents to write efficient serving infrastructure with no human intervention!
AI is now building the infrastructure it runs on. Getting real-time multimodal applications โ€” including live avatars, voice agents, video world models โ€” to actually run in real time takes serious manual systems work. Turns out a coding agent can do all the engineering work for you: up to 70ร— lower latency and 3.6ร— higher throughput, across 5 real-time apps. Today we release FlashRT โšก, an agent harness for real-time systems implementations and optimization. The developer writes only a simple reference spec, and the agent handles the rest end-to-end, producing efficient deployments across a range of GPU budgets in one shot, with no human intervention. โœ… Validated on 5 real-time applications that combine interactive video generation, text-to-speech, LLMs, etc. โœ… Runs on both NVIDIA and AMD GPUs; the agent optimizes for the hardware youโ€™re on โœ… Can easily compose with third-party serving engines, the agent automatically implements orchestration ๐Ÿ“„ Paper: arxiv.org/abs/2607.18171 ๐ŸŒ Website: infini-ai-lab.github.io/flasโ€ฆ ๐Ÿ”— GitHub: github.com/Infini-AI-Lab/Flaโ€ฆ ๐Ÿงต1/5
1
1
4
167
Krish Agarwal retweeted
๐Ÿ“ขAI is taking over the infrastructure it runs onโ€”and already doing better. FlashRT achieves up to 70ร— lower latency and 3.6ร— higher throughput across five real-time apps, even delivering stronger results on less human-optimized AMD GPUs than on NVIDIA. We used to handcraft static systems, but human expertise doesnโ€™t scale. In the agent era, every application should have an end-to-end system customized by AI for its workload and hardware. FlashRT is an early step toward that future.
AI is now building the infrastructure it runs on. Getting real-time multimodal applications โ€” including live avatars, voice agents, video world models โ€” to actually run in real time takes serious manual systems work. Turns out a coding agent can do all the engineering work for you: up to 70ร— lower latency and 3.6ร— higher throughput, across 5 real-time apps. Today we release FlashRT โšก, an agent harness for real-time systems implementations and optimization. The developer writes only a simple reference spec, and the agent handles the rest end-to-end, producing efficient deployments across a range of GPU budgets in one shot, with no human intervention. โœ… Validated on 5 real-time applications that combine interactive video generation, text-to-speech, LLMs, etc. โœ… Runs on both NVIDIA and AMD GPUs; the agent optimizes for the hardware youโ€™re on โœ… Can easily compose with third-party serving engines, the agent automatically implements orchestration ๐Ÿ“„ Paper: arxiv.org/abs/2607.18171 ๐ŸŒ Website: infini-ai-lab.github.io/flasโ€ฆ ๐Ÿ”— GitHub: github.com/Infini-AI-Lab/Flaโ€ฆ ๐Ÿงต1/5
1
23
4,975
Krish Agarwal retweeted
๐Ÿ“ขReleasing MonarchRT ๐Ÿฆ‹, an efficient architecture for real-time, interactive video generation. Butterfly structures and Monarch have been favorites in our early LLM explorations for linear layers @tri_dao (heard @OpenAI has been using it in 5.2 lately); video attention is their new home due to the intrinsic structure. Big potential for world models in gaming (Genie3-style) and robotics (DreamZeroโ€“style @DrJimFan). As systems get interactive, latency matters more than raw FPS โ€”and real efficiency gains on top of few-step AR diffusion without quality loss are still very hard. Check out MonarchRT for how we leverage video structure to make this possible. Congrats @InfiniAILab on new work in a new direction on Day 1 of Lunar New Year ๐Ÿงง ๐Ÿ’ช
Video generation models are improving fastโ€”real-time autoregressive models now deliver high quality at low latency, and theyโ€™re quickly being adopted for world models and robotics applications. So whatโ€™s the problem? Theyโ€™re still too slow on consumer hardware. ๐Ÿš€ What if we told you that we can get true real-time 16 FPS video generation on a single RTX 5090? (1.5-12x over FA 2/3/4 on 5090, H100, B200) Today we release MonarchRT ๐Ÿฆ‹, an efficient video attention that parameterizes attention maps as (tiled) Monarch matrices and delivers real E2E gains. ๐Ÿ“„ Paper: arxiv.org/abs/2602.12271 ๐ŸŒ Website: infini-ai-lab.github.io/Monaโ€ฆ ๐Ÿ”— GitHub: github.com/Infini-AI-Lab/Monโ€ฆ ๐Ÿงต1/n
1
10
1
96
15,772
Super excited to share the first work of my PhD! MonarchRT๐Ÿฆ‹ enables true real-time video generation on a single RTX 5090๐Ÿš€ This is one of the first sparse attention parameterizations to actually work for real-time video models. Happy Lunar New Year๐Ÿงง and Happy Mahashivratri ๐Ÿ”ฑ
Video generation models are improving fastโ€”real-time autoregressive models now deliver high quality at low latency, and theyโ€™re quickly being adopted for world models and robotics applications. So whatโ€™s the problem? Theyโ€™re still too slow on consumer hardware. ๐Ÿš€ What if we told you that we can get true real-time 16 FPS video generation on a single RTX 5090? (1.5-12x over FA 2/3/4 on 5090, H100, B200) Today we release MonarchRT ๐Ÿฆ‹, an efficient video attention that parameterizes attention maps as (tiled) Monarch matrices and delivers real E2E gains. ๐Ÿ“„ Paper: arxiv.org/abs/2602.12271 ๐ŸŒ Website: infini-ai-lab.github.io/Monaโ€ฆ ๐Ÿ”— GitHub: github.com/Infini-AI-Lab/Monโ€ฆ ๐Ÿงต1/n
2
2
12
684
Krish Agarwal retweeted
Finite space, infinite stories. "Aquarium" by @kikkawa_mese shows the power of PixVerse R1. Each generation breathes new life into the tank, creating a continuous, interactive loop of digital evolution. The era of "Infinite Flow" AI video is here. Rt+Reply+Follow = Invite Code (50 Random Pick)
93
112
5
274
435,190
Krish Agarwal retweeted
The whole @InfiniAILab is at #NeurIPS this week! Our group is currently working on diverse directions of GenAI, .e.g., Scalable and Efficient RL, VideoGen, Modeling, Model Arch & Sys Co-Design (Many new releases coming!!). Come and talk to us @RJ_Sadhukhan @IronSteveZhou @chenzhuoming911 @Xinyu2ML @haizhong_zheng @wonderingkrish @LiuAtlas89429
11
3
108
22,205
Krish Agarwal retweeted
๐Ÿฅณ Happy to share our new work โ€“ ย Kinetics: Rethinking Test-Time Scaling Laws ๐Ÿค”How to effectively build a powerful reasoning agent? Existing compute-optimal scaling laws suggest 64K thinking tokens + 1.7B model > 32B model. But, It only shows half of the picture! ๐Ÿšจ The O(Nยฒ) KV memory access in self-attention dominates the cost of test-time scaling (TTS). MoEs even worsen memory bottleneck by cutting compute. Our new scaling law Kinetics suggests investing in model size first before spending more in test-time compute. This insight leads to our next key finding โœจ Sparse Attention = Scalable TTS Our Kinetics sparse scaling law says that when doubling the resources, we should prioritize increasing test time tokens over attention density. โœ… 60+ pts improvement under the same compute budget โœ… 10ร— lower resource usage for equivalent performance โœ… Sparse attention becomes increasingly valuable in high-cost scenarios ๐Ÿ’กSparsity is key to unlocking full potential of TTS, because unlike pretraining, where scaling shows diminishing returns, TTS continues to benefit from increased token generation and more optimized inference paths. ๐Ÿ“„ Paper: arxiv.org/abs/2506.05333 ๐ŸŒ Website: infini-ai-lab.github.io/Kineโ€ฆ ๐Ÿ”— GitHub: github.com/Infini-AI-Lab/Kinโ€ฆ #ML #AI #LLM #Transformers #ScalingLaws #SparseAttention #Inference
7
68
16
243
84,391
Krish Agarwal retweeted
Presenting HadaCore: Tensor Core Accelerated Hadamard Transform Kernel ๐Ÿ” Take a look at how we achieve state-of-the-art performance on NVIDIA A100 & H100 GPUs over Dao AI Labโ€™s Fast Hadamard Transform Kernel ๐Ÿš„ Peak speedups of 3.6xโ€”read more: hubs.la/Q02-2q_G0 #AI
2
28
2
145
34,333