In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-175B, run faster in two weeks. Four years and 3,000 contributors later, that project is @vllm_project, one of the largest open-source inference engines, running on ~500,000 GPUs at any given time. He and fellow researchers Simon Mo, Kaichao You, and Roger Wang borrowed "virtual memory" from operating systems, handing out memory only as it's needed so each GPU can juggle requests of wildly different sizes. (The "v" in vLLM is for virtual.) Their first user, Berkeley neighbor @arena, cut its GPU needs in half. Today Google, Amazon, and Meta all run vLLM in production. In January, the core maintainers founded @inferact and raised $150M to give vLLM a full-time team. They're hiring across eng, marketing, and devrel in SF and Singapore: a16zjobs.substack.com/p/infe… @simon_mo_ @woosuk_k @KaichaoYou @rogerw0108

AWS CEO Matt Garman with a16z's Raghu Raghuram on the biggest cloud provider in the world being rebuilt for the agentic era: $220B in CapEx for 2026, two million NVIDIA GPUs on order, AWS's AI chips sold out through next year, and a new 30-second AWS account with no credit card required so agents can get rolling. 0:55 $170B in revenue, growing 37% 2:40 30-40% of revenue started as startups 4:55 Startups want a cloud for agents 9:10 An AWS account in 30 seconds 12:25 Agents don't need bulletproof databases 16:30 Why AWS won't sell every GPU to the frontier labs 22:00 Why Matt isn't worried about a bubble 24:45 Why AWS builds its own power plants 27:45 Planning for the next supply shortage 30:25 Saving one county $5K per resident 33:10 How AWS built its own chips 40:05 What's holding enterprise agents back 45:35 Why Bedrock never sends your data to the model 49:30 AWS's new AI security tool 51:50 Why AWS teams of 10 are now 3-4 YouTube: youtube.com/watch?v=rn_afJaP… @mattsgarman @awscloud @RaghuRaghuram

Vals AI co-founder and CEO Rayan Krishnan with a16z's Ben Horowitz and Jennifer Li on grading AI, what it costs, and who gets to make the rules: Every big industry eventually grows an independent testing layer. AI has credit ratings to learn from and Enron to avoid. Model capability today is still mostly self-reported. As public benchmarks saturate and models get better at optimizing for the tests themselves, Rayan makes the case for independent, continuously evolving evaluations. The harder problem is geopolitical. Reagan's "trust but verify" worked during the Cold War because you could fly over and count the missiles. No simple equivalent for AI models exists. In this conversation with Erik Torenberg, they get into how you measure a model's ability to improve itself, why every good benchmark eventually has to be retired, and what happens when token spend begins to rival employee salaries. 00:00 Intro 02:20 Llama 4 on public vs private benchmarks 05:24 Nobody agreed how to test humans either 06:55 What movie ratings teach us about AI 08:55 The Enron problem in benchmarking 11:36 Why a good benchmark has to be retired 13:22 Evals that run for weeks, not seconds 16:20 Where the real workday starts at 4pm 18:08 A firm really is just its evals 20:35 Why Sonnet can cost more than Opus 22:42 One engineer, 6 billion tokens in a day 25:05 Who should set the rules for models 28:55 Public sector enforces, private verifies 33:32 Why sovereign AI is inefficient and happening anyway 35:00 The AI version of trust-but-verify 37:15 Where cyber evals have to go next YouTube: youtube.com/watch?v=WO9c9qxD… @RayanKrishnan @ValsAI @bhorowitz @JenniferHli @eriktorenberg

OpenAI's Mark Sellke and Mehtaab Sawhney with a16z's Lisha Li, on the state of AI and mathematics: Before OpenAI released GPT‑6 Astra last week, the model was already doing original mathematics. Recorded before the launch, this conversation tells the story of how it got there. It began with GPT‑5. Mehtaab Sawhney pasted in an Erdős problem still listed as open. Five minutes later, the model surfaced a paper that had solved it. The exercise eventually uncovered published solutions to 10 more problems thought to be open. Then Astra went further. Told to "go have fun" with a high-dimensional sphere-packing problem, it improved a bound that had stood since the 1970s. Mehtaab had spent six months on the same problem in graduate school and made "absolutely zero progress." Another Astra result established that non-sofic groups exist with a roughly 15-page proof. A related human breakthrough took 250 pages and machinery from quantum complexity theory. OpenAI’s Mark Sellke and Mehtaab Sawhney join a16z’s Lisha Li on why wrong ideas pollute a human’s context window, why polished papers hide how mathematics is actually made, why a breakthrough can stop one prompt early, and what math rewards once proving stops being the bottleneck. 00:00 Intro 02:44 Cracking an Erdős problem in 5 minutes 06:20 Why a human quits and a model doesn't 08:50 Wrong ideas pollute your context window 11:45 Why math papers are bad training data 16:20 Nobody knows how to stack spheres in high dimensions 18:28 The orange-stacking proof 21:04 The 1970s Russian paper nobody could beat 24:14 The function that won a Fields Medal 27:09 How Astra beat the sphere-stacking record 29:20 Why error correction is sphere packing in disguise 35:15 The breakthrough Astra almost didn't bother with 39:22 Solving harder problems means it has better taste 41:00 One model for taste, one for the grind 44:40 Astra found an infinite group no finite one can imitate 52:00 250 pages of quantum complexity, or 15 of group theory 56:18 Only humans write 200-page proofs 1:00:38 What changes when proving stops being the bottleneck 1:02:44 The problems AI may never solve YouTube: youtube.com/watch?v=1JvyLGd2… @mehtaab_sawhney @MarkSellke @OpenAI @lishali88

We're thrilled to lead a $300M investment in Gimlet Labs. AI inference is one of the fastest-growing markets in the history of capitalism, and we are running out of nearly every physical input required to serve it. Inference demand compounds at software speed; power plants, data centers, and semiconductor fabs do not. We believe @gimletlabs has built the solution: the first multi-silicon inference cloud, designed to produce more intelligence from every watt. This system is already delivering up to 10x gains in throughput and interactivity on frontier models within the same power envelope. In a market starved for compute, efficiency is net-new capacity and latency is product differentiation. Wherever the existing stack ends, the Gimlet team starts. Welcome, @zainasgar, Michelle Nguyen, @oazizi, @nserrino, James Bartlett, and the Gimlet Labs team! By @RaghuRaghuram, @sarahdingwang, @shangdaxu, @steph_zhang