AI infrastructure that developers love 💚 Run inference, sandboxes, batch processing, training, and many other things on Modal

New York City
Joined July 2022
Pinned Tweet
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel. Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year. We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.
85
105
112
1,233
306,647
When several runs share the same GPUs, each one costs a fraction of what it would alone. Across math RL, a multi-turn search agent, and 256K-context SWE tasks, Spindle came in 20–80% cheaper per token than running the same scripts on Tinker.
1
1
7
461
Tinker changed how people do post-training. We're big fans, and so are many of the teams we work with. But as those teams scale up, they keep raising the same two problems: per-token pricing gets expensive quickly, and you’re limited to the models and features the provider decides to support. We think a good API deserves more than one implementation. Today we're releasing Spindle, an open-source, Tinker-compatible training server that runs in your Modal workspace. Your existing Tinker scripts run unchanged, and because you own the server, you pay for GPU time instead of tokens.
3
9
9
119
5,857
We demonstrate stable and correct training by reproducing Training Search Agents with GRPO by @lu__jasper, which trains gpt-oss-20b to perform multi-turn agentic search over a corpus of SEC filings with keyword-matching, grep, and read tools.
1
9
470
Long-horizon agentic RL is notoriously generation-bound, leading to low trainer utilization even with an optimized async RL loop. Our multi-LoRA backend enables high GPU efficiency by interleaving different clients’ training requests while waiting for long rollouts to complete.
1
6
354
Can LLM agents contribute to frontier computer science research? FrontierCS 2 gives a look into what agents can do as CS researchers and how much still remains open. Powered by Modal.
FrontierCS 2 is our new benchmark for CS research & development. Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
8
6
50
2,606
Performant reinforcement learning is a hard infrastructure and research problem. We’ve recently made on-demand multi-node clusters available and showed how to take advantageous of heterogeneous sampling compute. But there still exists the need for high-fidelity environments, clear and robust reward functions, and tuned training recipes that actually work. Most teams are actually nearly there! With an agent harness deployed in production and evals/agent traces collected from that deployment, you just need an easy way to start training your data. So we built a training library that gives you the building blocks, tuned defaults, and observability you need for easy, production-grade LLM training. It’s available now. All it takes is a prompt.
7
19
7
121
10,524
‍@AfterQuery curates long horizon data from domain experts for models to be trained on. Using our training library, they were able to kick off autoresearch for models like Kimi and DeepSeek at 256k context length on their existing datasets.
1
1
17
1,168
✅ 96% of issues resolved before customer impact. ✅ 99% of incidents detected within five minutes. Read the full story to see how the team at @modal use Datadog to operate AI infrastructure across 20+ cloud providers, launching 26M containers every day while maintaining unified visibility across the platform: bit.ly/4eLDmrh
3
5
17
7,861
Voice agents, multiplayer games, and other workloads need to maintain continuity between requests. With Sticky Sessions, built on Modal's serverless Servers, you can route requests in a session to the same container and autoscale with demand. Here's a look at how we built it:
5
2
45
3,301
Congrats to @GradiumAI on launching Phonon! We built a voice agent with their new TTS models and Pipecat on Modal to put them to work. Read more about Phonon and check out the demo: gradium.ai/phonon
A new frontier for on-device speech. We're open-sourcing Phonon, a suite of TTS models under 50m parameters, setting a new standard for naturalness and accuracy across languages, with the broadest inference support to date. Phonon-31m is available under Apache 2.0. gradium.ai/phonon🧵
2
5
42
2,273
Replying to @modal
@modal and @GrayAreaorg are partnering to put the "art" back in "artificial intelligence". We've sponsored a dozen projects by leading digital artists that will be shown at Gray Area next week. Come thru on Thursday for a panel featuring @ideeplistening and @Ryphi10. Register here: luma.com/modal-48ev
1
8
36
2,760
More than 40x the parameters of Mercury 1, still running at 1,000+ tokens/sec. Congrats to the @_inception_ai team on Mercury 3!
1/4 Today, we’re releasing Mercury 3. Mercury 3 is by far our most powerful model and the most intelligent diffusion LLM ever released. It outperforms GPT-6 Luna and Haiku 5.5 on quality while running at 1000+ tok/sec.
6
6
2
136
8,403
We got our hands on @nvidia's new Vera CPU and put it to the test. We built a benchmark using real-world agentic workloads to see how it performs under contention. In early tests, Vera delivered up to 2x the throughput of comparable x86 hardware, maintaining low latency even as hundreds of concurrent workloads competed for resources. For Modal Sandboxes, that could mean running more agents in parallel without sacrificing latency.
29
26
18
224
26,925
When we introduced VM Sandboxes, we wanted to preserve Modal’s sub-second cold starts without compromising performance. We ended up building a new filesystem stack that dramatically improved performance, and wrote about the engineering challenges and decisions that got us there.
The secret to Modal's speed is our custom lazily loaded container filesystem, ImageFS. ImageFS is built on a massive tiered distributed filesystem, layering local NVMe disks, regional and p2p caches, and global blob stores. When we launched VM Sandboxes this month, we wanted to make sure they could take advantage of ImageFS in the same way our gVisor containers do. To do that, we built out a new filesystem stack that tricks a VM's ext4 driver into thinking it's talking to a normal disk.
5
9
1
143
7,986
Modal retweeted
1/ I trained an AI to be top 1% in the world at Super Smash Bros. Melee by watching 80k hours of human replay. Meet HAL.
148
120
96
1,860
370,995