
ECE PhD student @CarnegieMellon
Pittsburgh, PA
Joined April 2016
- Tweets12
- Following101
- Followers46
- Likes206
Krish Agarwal retweeted
Replying to @bespokelabsai
Congrats on the release of Multi-Agent Coordination Benchmark @bespokelabsai! ๐ฅ
We actually came across the same observations in our recent Singularity-Bench (Released on Monday)! In a task to design an efficient large-scale serving system, Opus 5.5 actually managed to improve its score when required to use subagents, but GPT 6.1 Sol actually degraded its score when required to use subagents.๐คฏ
In our case, solving the task effectively requires the agent to completely revamp the starter implementation, but when required to use subagents, GPT 6.1 Sol chose to just use separate agents to make fine-grained edits (local optimization) to the starter code, suggesting that it has a harder time with multi-agent coordination! ๐
Check out our work here: xitter.gq/InfiniAILab/status/212โฆ
Introducing Singularity-Bench ๐ณ๏ธ
๐ We measure whether agents can autonomously conduct the full agent development lifecycle: six long-horizon, end-to-end tasks spanning pre-, mid-, and post-training, inference engines, cluster management, and harness development on production GB300 clusters!
We emphasize rigorous verification to guarantee correctness and execution integrity ๐ฎโโ๏ธ
๐ Big shoutout to all authors contributing from academia and industry: @IronSteveZhou, @wonderingkrish, @chenzhuoming911, Zerui Wang, @KyriectionZhang, Wenshuang Li, Xinpeng Wei, Zhichen Zeng, @Jingyu227, @JueWANG26088228, @Wu_Sen, Ziheng Jiang, @zhendongucb, @hancai_hm, @Hades317, @ying11231, @ce_zhang, @uiuc_aisecure, and @BeidiChen!
How far are current agents from building their successors end to end? ๐ค We're here to find out!
๐งต1/n
๐ Website: infini-ai-lab.github.io/singโฆ
Excited to release Singularity-Bench! Opus 5.5 might not be able to autonomously develop its successor just yet, but it seems to be getting close ๐
Introducing Singularity-Bench ๐ณ๏ธ
๐ We measure whether agents can autonomously conduct the full agent development lifecycle: six long-horizon, end-to-end tasks spanning pre-, mid-, and post-training, inference engines, cluster management, and harness development on production GB300 clusters!
We emphasize rigorous verification to guarantee correctness and execution integrity ๐ฎโโ๏ธ
๐ Big shoutout to all authors contributing from academia and industry: @IronSteveZhou, @wonderingkrish, @chenzhuoming911, Zerui Wang, @KyriectionZhang, Wenshuang Li, Xinpeng Wei, Zhichen Zeng, @Jingyu227, @JueWANG26088228, @Wu_Sen, Ziheng Jiang, @zhendongucb, @hancai_hm, @Hades317, @ying11231, @ce_zhang, @uiuc_aisecure, and @BeidiChen!
How far are current agents from building their successors end to end? ๐ค We're here to find out!
๐งต1/n
๐ Website: infini-ai-lab.github.io/singโฆ
Krish Agarwal retweeted
๐ง How can we keep scaling LLMs with minimal changes in their serving energy?
DeepSeekโs Engram and our concurrent work STEM showed the promise of static sparse memory: large embedding tables with efficient access. But can mature pretrained models learn to use a newly added memory within a limited training budget, avoiding costly pretraining from scratch?
๐ Introducing STAPLE: a recipe to extend pretrained models' abilities with static sparse memory. Serving energy per token stays close to dense and >50% below MoE upcycling in high-throughput regime.
Scalable capacity, interpretable memory, and compatibility with MoE upcycling - without any meaningful increase in serving cost. ๐งต
Paper: arxiv.org/abs/2610.12599
Code: github.com/Infini-AI-Lab/STAโฆ
Website: infini-ai-lab.github.io/STAPโฆ
Krish Agarwal retweeted
How well can agents continually self-improve from serving experience? Weโre excited to introduce our new benchmark, ๐๐๐ซ๐ฏ๐๐๐๐๐ซ๐ง๐๐๐ง๐๐ก:
Real-world deployment is not a one-time test of capability. The knowledge agents need is often hidden and can change over time.
ServeLearnBench tests whether agents can infer, apply, and revise this knowledge from interaction.
Across 5 learning harnesses ร 6 models, we find:
1. ๐๐๐ซ๐ ๐ ๐ฅ๐๐๐ซ๐ง๐ข๐ง๐ ๐ ๐๐ฉ๐ฌ ๐ซ๐๐ฆ๐๐ข๐ง. Agents can solve tasks when the hidden policy is given, but struggle to discover it from experience.
2. ๐๐จ ๐๐ซ๐๐ ๐ฅ๐ฎ๐ง๐๐ก ๐๐จ๐ซ ๐๐๐๐ฉ๐ญ๐๐ญ๐ข๐จ๐ง. Continual learning can be costlyโand can even degrade already-correct behavior.
3. ๐๐ฑ๐ฉ๐ฅ๐จ๐ซ๐๐ญ๐ข๐จ๐ง ๐ข๐ฌ ๐ ๐ค๐๐ฒ ๐๐จ๐ญ๐ญ๐ฅ๐๐ง๐๐๐ค. Better exploration strongly correlates with better adaptation.
Happy to share our recent work on FlashRT โก
Multimodal applications are highly diverse. Most systems work still falls on human developers, which is unscalable.
FlashRT automates this by guiding coding agents to write efficient serving infrastructure with no human intervention!
AI is now building the infrastructure it runs on.
Getting real-time multimodal applications โ including live avatars, voice agents, video world models โ to actually run in real time takes serious manual systems work. Turns out a coding agent can do all the engineering work for you: up to 70ร lower latency and 3.6ร higher throughput, across 5 real-time apps.
Today we release FlashRT โก, an agent harness for real-time systems implementations and optimization. The developer writes only a simple reference spec, and the agent handles the rest end-to-end, producing efficient deployments across a range of GPU budgets in one shot, with no human intervention.
โ
Validated on 5 real-time applications that combine interactive video generation, text-to-speech, LLMs, etc.
โ
Runs on both NVIDIA and AMD GPUs; the agent optimizes for the hardware youโre on
โ
Can easily compose with third-party serving engines, the agent automatically implements orchestration
๐ Paper: arxiv.org/abs/2607.18171
๐ Website: infini-ai-lab.github.io/flasโฆ
๐ GitHub: github.com/Infini-AI-Lab/Flaโฆ
๐งต1/5
Krish Agarwal retweeted
๐ขAI is taking over the infrastructure it runs onโand already doing better. FlashRT achieves up to 70ร lower latency and 3.6ร higher throughput across five real-time apps, even delivering stronger results on less human-optimized AMD GPUs than on NVIDIA.
We used to handcraft static systems, but human expertise doesnโt scale. In the agent era, every application should have an end-to-end system customized by AI for its workload and hardware.
FlashRT is an early step toward that future.
AI is now building the infrastructure it runs on.
Getting real-time multimodal applications โ including live avatars, voice agents, video world models โ to actually run in real time takes serious manual systems work. Turns out a coding agent can do all the engineering work for you: up to 70ร lower latency and 3.6ร higher throughput, across 5 real-time apps.
Today we release FlashRT โก, an agent harness for real-time systems implementations and optimization. The developer writes only a simple reference spec, and the agent handles the rest end-to-end, producing efficient deployments across a range of GPU budgets in one shot, with no human intervention.
โ
Validated on 5 real-time applications that combine interactive video generation, text-to-speech, LLMs, etc.
โ
Runs on both NVIDIA and AMD GPUs; the agent optimizes for the hardware youโre on
โ
Can easily compose with third-party serving engines, the agent automatically implements orchestration
๐ Paper: arxiv.org/abs/2607.18171
๐ Website: infini-ai-lab.github.io/flasโฆ
๐ GitHub: github.com/Infini-AI-Lab/Flaโฆ
๐งต1/5
Krish Agarwal retweeted
๐ขReleasing MonarchRT ๐ฆ, an efficient architecture for real-time, interactive video generation. Butterfly structures and Monarch have been favorites in our early LLM explorations for linear layers @tri_dao (heard @OpenAI has been using it in 5.2 lately); video attention is their new home due to the intrinsic structure.
Big potential for world models in gaming (Genie3-style) and robotics (DreamZeroโstyle @DrJimFan). As systems get interactive, latency matters more than raw FPS โand real efficiency gains on top of few-step AR diffusion without quality loss are still very hard.
Check out MonarchRT for how we leverage video structure to make this possible. Congrats @InfiniAILab on new work in a new direction on Day 1 of Lunar New Year ๐งง ๐ช
Video generation models are improving fastโreal-time autoregressive models now deliver high quality at low latency, and theyโre quickly being adopted for world models and robotics applications. So whatโs the problem? Theyโre still too slow on consumer hardware.
๐ What if we told you that we can get true real-time 16 FPS video generation on a single RTX 5090? (1.5-12x over FA 2/3/4 on 5090, H100, B200)
Today we release MonarchRT ๐ฆ, an efficient video attention that parameterizes attention maps as (tiled) Monarch matrices and delivers real E2E gains.
๐ Paper: arxiv.org/abs/2602.12271
๐ Website: infini-ai-lab.github.io/Monaโฆ
๐ GitHub: github.com/Infini-AI-Lab/Monโฆ
๐งต1/n
Super excited to share the first work of my PhD! MonarchRT๐ฆ enables true real-time video generation on a single RTX 5090๐
This is one of the first sparse attention parameterizations to actually work for real-time video models.
Happy Lunar New Year๐งง and Happy Mahashivratri ๐ฑ
Video generation models are improving fastโreal-time autoregressive models now deliver high quality at low latency, and theyโre quickly being adopted for world models and robotics applications. So whatโs the problem? Theyโre still too slow on consumer hardware.
๐ What if we told you that we can get true real-time 16 FPS video generation on a single RTX 5090? (1.5-12x over FA 2/3/4 on 5090, H100, B200)
Today we release MonarchRT ๐ฆ, an efficient video attention that parameterizes attention maps as (tiled) Monarch matrices and delivers real E2E gains.
๐ Paper: arxiv.org/abs/2602.12271
๐ Website: infini-ai-lab.github.io/Monaโฆ
๐ GitHub: github.com/Infini-AI-Lab/Monโฆ
๐งต1/n
Finite space, infinite stories.
"Aquarium" by @kikkawa_mese shows the power of PixVerse R1.
Each generation breathes new life into the tank, creating a continuous, interactive loop of digital evolution.
The era of "Infinite Flow" AI video is here.
Rt+Reply+Follow = Invite Code (50 Random Pick)
Krish Agarwal retweeted
The whole @InfiniAILab is at #NeurIPS this week! Our group is currently working on diverse directions of GenAI, .e.g., Scalable and Efficient RL, VideoGen, Modeling, Model Arch & Sys Co-Design (Many new releases coming!!). Come and talk to us @RJ_Sadhukhan @IronSteveZhou @chenzhuoming911 @Xinyu2ML @haizhong_zheng @wonderingkrish @LiuAtlas89429
Krish Agarwal retweeted
๐ฅณ Happy to share our new work โ ย Kinetics: Rethinking Test-Time Scaling Laws
๐คHow to effectively build a powerful reasoning agent?
Existing compute-optimal scaling laws suggest 64K thinking tokens + 1.7B model > 32B model.
But, It only shows half of the picture!
๐จ The O(Nยฒ) KV memory access in self-attention dominates the cost of test-time scaling (TTS).
MoEs even worsen memory bottleneck by cutting compute.
Our new scaling law Kinetics suggests investing in model size first before spending more in test-time compute.
This insight leads to our next key finding
โจ Sparse Attention = Scalable TTS
Our Kinetics sparse scaling law says that when doubling the resources, we should prioritize increasing test time tokens over attention density.
โ
60+ pts improvement under the same compute budget
โ
10ร lower resource usage for equivalent performance
โ
Sparse attention becomes increasingly valuable in high-cost scenarios
๐กSparsity is key to unlocking full potential of TTS, because unlike pretraining, where scaling shows diminishing returns, TTS continues to benefit from increased token generation and more optimized inference paths.
๐ Paper: arxiv.org/abs/2506.05333
๐ Website: infini-ai-lab.github.io/Kineโฆ
๐ GitHub: github.com/Infini-AI-Lab/Kinโฆ
#ML #AI #LLM #Transformers #ScalingLaws #SparseAttention #Inference
Krish Agarwal retweeted
Presenting HadaCore: Tensor Core Accelerated Hadamard Transform Kernel ๐ Take a look at how we achieve state-of-the-art performance on NVIDIA A100 & H100 GPUs over Dao AI Labโs Fast Hadamard Transform Kernel ๐ Peak speedups of 3.6xโread more: hubs.la/Q02-2q_G0 #AI










