Writing and talking @substratemag, prev @Etched

San Francisco, CA
Joined July 2019
Pinned Tweet
I sat down with @dylan522p and spoke with over a dozen people about his information machine. "I don't think anyone who has a traditional equity research background would even try to figure out where $3T in [the AI labs' combined] ARR comes from." Slack private messaging limits, finding CoWoS volume, and the industry they're looking at next - I wrote about SemiAnalysis here: substratemag.com/dylanpatel/…
44
51
16
723
308,092
CAT retweeted
Everyone in the Valley has a take on whether China is winning AI, robotics, and biotech. Almost nobody understands how the capital works Notes from meeting China's tier-1 investors and founders on how their venture ecosystem actually works.
81
137
69
873
364,294
I’m hiring a full time creative operator I made this video in 48 hours, I will teach you how Must enjoy talking to AI researchers and hollywood cinematographers. In-person 6 days a week in SF. Get equity in top companies. 5k referral bonus. Duties may include:
Last month, we demonstrated high-rate alchemy: turning mercury into real, stable gold, beating CERN’s record by a factor of 10. Last week, we enriched isotopes in the world’s most powerful plasma centrifuge, beating the Soviet record that stood for 40 years. Now, we’re building a fusion reactor in San Francisco to achieve record power density.
43
29
2
356
24,350
It's near-impossible to outbid the HBM shortage, but sidestepping it is a different question GDDR usage, air-cooled PCIe cards, and alternative vendors can all help work around supply chain constraints
I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale. "We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand." Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.
1
7
362
the conventional view is that datacenters require reliability but if your workload tolerated 95% uptime? 90%? you could move workloads between sites if the dc drops and take advantage of underutilized (cheaper) capacity
I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale. "We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand." Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.
5
3
1
45
5,409
On modular datacenters: NVIDIA has been piloting modular (5-20mw) datacenters, and SemiAnalysis predicts they'll make up 31% of capacity by 2028. A big reason why is lead time. 50mw-100mw datacenter lead times are 12-24 months. For <1mw datacenters, 16 weeks
I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale. "We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand." Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.
3
10
762
I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale. "We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand." Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.
34
25
6
339
38,564
24GW for datacenters in just China is insane. That's almost as much as Europe, the Middle East, Africa, and the rest of the Asia-Pacific combined The US is still over twice that at 56GW but buildouts here take >2x as long (China is doing <12 mo for 100MW) and we have almost <3x China's power capacity, which is a gap that's much harder to close
The Chinese AI Infrastructure Boom: Introducing the SemiAnalysis China Datacenter Model 1,000+ facilities across 60+ operators mapped, built retail-first and flipped by AI, largest hyperscaler leases 1/5 national capacity, 100MW in 12 months, Eastern Data Western Compute newsletter.semianalysis.com/…
2
2
1
33
3,452
In SF, $3T combined frontier lab ARR by 2030 is possible If you believe this, you’ll have different priors than everyone else. If it’s true, there’s a lot of money on the table More @substratemag
Replying to @SemiAnalysis_
@SemiAnalysis_ has be AI-pilled to have the right priors A few years ago, no one thought demand would be able to hit today's numbers In 2023, while modeling datacenter builds, one team member said, "There was a thought of 'Did we mess anything up on the numbers?'....I asked the team, 'How confident are you in these numbers?' They were pretty damn confident, so I just worked on that assumption."
1
1
27
5,233
@evanjconrad once told me that if you stay in SF long enough, you’ll just figure it out The future is unevenly distributed, and it hits SF first. If you can see it, you can time the waves
1
9
855
Replying to @SemiAnalysis_
@SemiAnalysis_'s early bet on predicting NVIDIA's revenue via TSMC CoWoS output and earnings calls from tool companies kickstarted their model program The accelerator model is one of the most popular today
2
2
7
698
More coming via @substratemag!
13
8,524
We’ve raised $80 million in Series A funding, co-led by the Valor Atreides AI Fund and Hummingbird Ventures, with continued participation from Conviction, Abstract, A*, and Grant Gordon. This brings our total raised to over $100 million. We started Watney by asking a simple question -- what can a robot enable in industries where the work remains valuable on an infinite horizon? We found an answer in the fundamental inputs to civilization: energy, matter, and intelligence. A robot will never be as charming as a barista or as personable as a housekeeper, but it can be more exacting than a surgeon. And, a breakthrough in one quickly becomes the capability of millions. We don’t imitate human motions or processes. We care about the new, differentiated capabilities robots can unlock. We work on problems where our choice of embodiment gives us an order of magnitude advantage, whether through precision, reliability, or scale. Compute is just beginning to transform society, but it’s constrained by execution. Since 2025, Watney has been serving the largest hyperscalers in the world, helping accelerate their compute rollout through an end-to-end deployment model. Across hundreds of thousands of hours in customer facilities, our systems have achieved more than four nines of reliability. They’ve enabled our customers to build data centers faster, at larger scales, and with more ambitious architectures. Today, we operate the largest fleet of dexterous robots running 24/7/365 across the United States. This is all to say, it's an exciting time at Watney. We’re all here because we want to see more ambition in the physical world. We are hiring across all domains. Join us.
81
60
65
727
283,956
> most powerful model yet > previous families all somewhat misaligned (internal astra family research model, internal sol family research model, public sol) > openai claims astra is better aligned, prevalence of behavior with “severity 3+” flags very low (0.052%) > actuallydifferent?.jpg > monitorability decreased > astra can control its own CoT > astra can automatically evade monitoring when being monitored > does not seem to be in architecture, CoT control increases over RL > 😐.jpg models are not “aligned” or “misaligned” on a binary; they are probabilistically aligned. monitorability matters because at the tails, misalignment is almost inevitable, but monitoring is not
GPT-6 Astra is more aligned than our previous models. But it’s also less monitorable, which is a concerning trend that we take very seriously. We believe monitorability drop comes from a jump in intelligence and not direct optimization pressure on CoT or architecture changes. More thoughts in the thread.
10
3,277
It's very strange that there's been limited mainstream media coverage on the most important AI attack in history. I wrote a relatively lay summary on the attack, why it matters, and training/policy discussion.
19
4
1
40
5,258
Thank you to @theojaffee, @mgrczyk, and other friends for reading and providing feedback on drafts! The frontier is wild, and further feedback is appreciated. DMs open to those interested in discussing this piece and related topics.
5
838
People often ask why today's tech wealth doesn't go towards more public works, like libraries, universities, and museums during the late 1800s. As far as I can tell the answer is that they would literally not be allowed to be built. In 2014, George Lucas, a long-time Bay Area resident who built Lucasfilm in the Bay, wanted to build a museum in the Presidio as a gift to the people of San Francisco, to be personally funded to the tune of ~$700 million. The Presidio Trust rejected it. He then tried to build it in his wife's hometown of Chicago, but local nonprofit Friends of the Parks sued and held the project up for two years, long enough for Lucas to give up on the city. The Lucas Museum of Narrative Art will finally open in Los Angeles next month after more than a decade of planning, eight years of construction, and over $1 billion personally spent by Lucas. Meanwhile, San Francisco has built no major museum projects since the 2016 expansion of SFMOMA, and no new libraries since the North Beach Branch in 2014. Public works, like so many other things, are bottlenecked not by donor will or available funding but by vetocracy
164
857
89
9,632
1,310,140
Neil's take is controversial in light of the OAI release (Jalapeño targets latency/W) but it's a great prediction for inference use in an open-source-friendly regime Labs optimize for latency/W, can scale up chip volume, and capture value on frontier intelligence. Enterprises on open-source optimize for cost-efficient throughput (TPS or tok/$) and use the affordable middle class of intelligence for background work and long-horizon tasks/agents
Neil Movva (@neilmovva) started his career at Nvidia, working on GPUs and kernels, and has an unusually deep understanding of inference, from software to chips to power. We spend a lot of time on each of those layers, how they connect, and where the important tradeoffs are. What makes this conversation special is how detailed it is (like a 401-level class), yet Neil makes it remarkably clear and easy to follow. Today he runs Sail Research, a company building infrastructure for agents to make tokens as cheap as possible. We discuss: - Latency versus throughput - Why there are no bad chips, only bad pricing - The end of kernel engineering - Buying chips and power no one else wants - New chip architectures - Nvidia lore + his contrarian view of the company - Open source and the frontier labs I learned a ton. Enjoy! TIMESTAMPS 0:00 Intro 0:38 Building a “Token Factory” 4:21 The Future of Background Agents 13:09 Nvidia and the GPU Stack 23:27 Chips, Memory, and Transformers 36:14 The Future of AI Training Data 44:32 Chip Scarcity and Compute Arbitrage 52:44 Reinventing the AI Data Center 59:01 Power and the “Scavenger Strategy” 1:10:10 Open vs. Closed AI
1
14
3,813