
Writing and talking @substratemag, prev @Etched
San Francisco, CA
Joined July 2019
- Tweets294
- Following660
- Followers1.7K
- Likes13.9K
Pinned Tweet
I sat down with @dylan522p and spoke with over a dozen people about his information machine.
"I don't think anyone who has a traditional equity research background would even try to figure out where $3T in [the AI labs' combined] ARR comes from."
Slack private messaging limits, finding CoWoS volume, and the industry they're looking at next - I wrote about SemiAnalysis here: substratemag.com/dylanpatel/…
Everyone in the Valley has a take on whether China is winning AI, robotics, and biotech. Almost nobody understands how the capital works
Notes from meeting China's tier-1 investors and founders on how their venture ecosystem actually works.
CAT retweeted
I’m hiring a full time creative operator
I made this video in 48 hours, I will teach you how
Must enjoy talking to AI researchers and hollywood cinematographers. In-person 6 days a week in SF. Get equity in top companies. 5k referral bonus. Duties may include:
Last month, we demonstrated high-rate alchemy: turning mercury into real, stable gold, beating CERN’s record by a factor of 10.
Last week, we enriched isotopes in the world’s most powerful plasma centrifuge, beating the Soviet record that stood for 40 years.
Now, we’re building a fusion reactor in San Francisco to achieve record power density.
CAT retweeted
It's near-impossible to outbid the HBM shortage, but sidestepping it is a different question
GDDR usage, air-cooled PCIe cards, and alternative vendors can all help work around supply chain constraints
I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale.
"We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand."
Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.
the conventional view is that datacenters require reliability
but if your workload tolerated 95% uptime? 90%? you could move workloads between sites if the dc drops and take advantage of underutilized (cheaper) capacity
I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale.
"We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand."
Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.
CAT retweeted
On modular datacenters:
NVIDIA has been piloting modular (5-20mw) datacenters, and SemiAnalysis predicts they'll make up 31% of capacity by 2028.
A big reason why is lead time. 50mw-100mw datacenter lead times are 12-24 months. For <1mw datacenters, 16 weeks
I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale.
"We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand."
Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.
I interviewed @neilmovva for @substratemag on latency-insensitive inference, alternative chips, and why massive datacenters face diseconomies of scale.
"We've always predicted that async inference is going to eat the world. That could easily be 95% or more of inference demand."
Neil's been at the frontier of inference for years, from NVIDIA to co-founding @sailresearchco. I learned a lot from our conversation - you can listen below.
Links:
Youtube: youtube.com/watch?v=2w-I4pAS…
Spotify: open.spotify.com/episode/1n5…
Transcript: read.substratemag.com/p/neil…
24GW for datacenters in just China is insane. That's almost as much as Europe, the Middle East, Africa, and the rest of the Asia-Pacific combined
The US is still over twice that at 56GW but buildouts here take >2x as long (China is doing <12 mo for 100MW) and we have almost <3x China's power capacity, which is a gap that's much harder to close
The Chinese AI Infrastructure Boom:
Introducing the SemiAnalysis China Datacenter Model
1,000+ facilities across 60+ operators mapped,
built retail-first and flipped by AI,
largest hyperscaler leases 1/5 national capacity, 100MW in 12 months, Eastern Data Western Compute
newsletter.semianalysis.com/…
In SF, $3T combined frontier lab ARR by 2030 is possible
If you believe this, you’ll have different priors than everyone else. If it’s true, there’s a lot of money on the table
More @substratemag
Replying to @SemiAnalysis_
@SemiAnalysis_ has be AI-pilled to have the right priors
A few years ago, no one thought demand would be able to hit today's numbers
In 2023, while modeling datacenter builds, one team member said, "There was a thought of 'Did we mess anything up on the numbers?'....I asked the team, 'How confident are you in these numbers?' They were pretty damn confident, so I just worked on that assumption."
@evanjconrad once told me that if you stay in SF long enough, you’ll just figure it out
The future is unevenly distributed, and it hits SF first. If you can see it, you can time the waves
CAT retweeted
Replying to @SemiAnalysis_
@SemiAnalysis_'s early bet on predicting NVIDIA's revenue via TSMC CoWoS output and earnings calls from tool companies kickstarted their model program
The accelerator model is one of the most popular today
CAT retweeted
We’ve raised $80 million in Series A funding, co-led by the Valor Atreides AI Fund and Hummingbird Ventures, with continued participation from Conviction, Abstract, A*, and Grant Gordon. This brings our total raised to over $100 million.
We started Watney by asking a simple question -- what can a robot enable in industries where the work remains valuable on an infinite horizon? We found an answer in the fundamental inputs to civilization: energy, matter, and intelligence.
A robot will never be as charming as a barista or as personable as a housekeeper, but it can be more exacting than a surgeon. And, a breakthrough in one quickly becomes the capability of millions. We don’t imitate human motions or processes. We care about the new, differentiated capabilities robots can unlock. We work on problems where our choice of embodiment gives us an order of magnitude advantage, whether through precision, reliability, or scale.
Compute is just beginning to transform society, but it’s constrained by execution. Since 2025, Watney has been serving the largest hyperscalers in the world, helping accelerate their compute rollout through an end-to-end deployment model.
Across hundreds of thousands of hours in customer facilities, our systems have achieved more than four nines of reliability. They’ve enabled our customers to build data centers faster, at larger scales, and with more ambitious architectures. Today, we operate the largest fleet of dexterous robots running 24/7/365 across the United States.
This is all to say, it's an exciting time at Watney. We’re all here because we want to see more ambition in the physical world. We are hiring across all domains. Join us.
> most powerful model yet
> previous families all somewhat misaligned (internal astra family research model, internal sol family research model, public sol)
> openai claims astra is better aligned, prevalence of behavior with “severity 3+” flags very low (0.052%)
> actuallydifferent?.jpg
> monitorability decreased
> astra can control its own CoT
> astra can automatically evade monitoring when being monitored
> does not seem to be in architecture, CoT control increases over RL
> 😐.jpg
models are not “aligned” or “misaligned” on a binary; they are probabilistically aligned. monitorability matters because at the tails, misalignment is almost inevitable, but monitoring is not
GPT-6 Astra is more aligned than our previous models. But it’s also less monitorable, which is a concerning trend that we take very seriously. We believe monitorability drop comes from a jump in intelligence and not direct optimization pressure on CoT or architecture changes. More thoughts in the thread.
It's very strange that there's been limited mainstream media coverage on the most important AI attack in history.
I wrote a relatively lay summary on the attack, why it matters, and training/policy discussion.
Thank you to @theojaffee, @mgrczyk, and other friends for reading and providing feedback on drafts! The frontier is wild, and further feedback is appreciated. DMs open to those interested in discussing this piece and related topics.
CAT retweeted
People often ask why today's tech wealth doesn't go towards more public works, like libraries, universities, and museums during the late 1800s. As far as I can tell the answer is that they would literally not be allowed to be built.
In 2014, George Lucas, a long-time Bay Area resident who built Lucasfilm in the Bay, wanted to build a museum in the Presidio as a gift to the people of San Francisco, to be personally funded to the tune of ~$700 million. The Presidio Trust rejected it. He then tried to build it in his wife's hometown of Chicago, but local nonprofit Friends of the Parks sued and held the project up for two years, long enough for Lucas to give up on the city. The Lucas Museum of Narrative Art will finally open in Los Angeles next month after more than a decade of planning, eight years of construction, and over $1 billion personally spent by Lucas.
Meanwhile, San Francisco has built no major museum projects since the 2016 expansion of SFMOMA, and no new libraries since the North Beach Branch in 2014. Public works, like so many other things, are bottlenecked not by donor will or available funding but by vetocracy
Neil's take is controversial in light of the OAI release (Jalapeño targets latency/W) but it's a great prediction for inference use in an open-source-friendly regime
Labs optimize for latency/W, can scale up chip volume, and capture value on frontier intelligence. Enterprises on open-source optimize for cost-efficient throughput (TPS or tok/$) and use the affordable middle class of intelligence for background work and long-horizon tasks/agents
Neil Movva (@neilmovva) started his career at Nvidia, working on GPUs and kernels, and has an unusually deep understanding of inference, from software to chips to power.
We spend a lot of time on each of those layers, how they connect, and where the important tradeoffs are.
What makes this conversation special is how detailed it is (like a 401-level class), yet Neil makes it remarkably clear and easy to follow.
Today he runs Sail Research, a company building infrastructure for agents to make tokens as cheap as possible.
We discuss:
- Latency versus throughput
- Why there are no bad chips, only bad pricing
- The end of kernel engineering
- Buying chips and power no one else wants
- New chip architectures
- Nvidia lore + his contrarian view of the company
- Open source and the frontier labs
I learned a ton. Enjoy!
TIMESTAMPS
0:00 Intro
0:38 Building a “Token Factory”
4:21 The Future of Background Agents
13:09 Nvidia and the GPU Stack
23:27 Chips, Memory, and Transformers
36:14 The Future of AI Training Data
44:32 Chip Scarcity and Compute Arbitrage
52:44 Reinventing the AI Data Center
59:01 Power and the “Scavenger Strategy”
1:10:10 Open vs. Closed AI



















