trying to manufacture inference hw + sw @fdotinc @scale_at_gmi
San Jose/Taipei
Joined December 2021
- Tweets1.9K
- Following7.1K
- Followers1.3K
- Likes36.2K
Pinned Tweet
I love inference! Tap in🧑🏻💻
Welcoming @solidwillity to SCALE! 🚀
His startup, Touchdown Labs, makes AI inference optimization simple and accessible for resource-constrained organizations.
Can't wait to see what you build!
#SCALEatGMI #GMICloud #Startup
William Chen retweeted
🚨🚨 There will be no memory oversupply, even beyond 2030
As you all know, HBM demand is enormous, and meeting that demand is constraining the production capacity available for conventional DRAM. HBM already requires far more wafers than conventional DDR5 to produce the same number of bits.
So what happens when new 3D stacked DRAM architectures, integrating DRAM directly with logic, begin to scale around 2029 or 2030?
Could that put even more pressure on conventional DRAM supply? As far as I know, the memory companies have not even finalized which DRAM process nodes they will use for these architectures. And if my understanding of the architecture is correct, 3D stacked DRAM could impose even greater constraints on bit output than the DRAM used in today’s HBM.
I think many analysts still aren’t even considering the supply constraints from 3D stacked DRAM. We’ll probably only start hearing about this next year.
Many companies in the industry are already collaborating with memory vendors on 3D stacked DRAM. These are things I already discussed in my articles two months ago, but I’m bringing them up again.
With SK hynix posting strong results, this seems like a good time to revisit the topic 😊
William Chen retweeted
🚨🇨🇳 A Chinese glass company just turned the RIOTS in France into FREE advertising!
Hengxinfeng filmed NOTHING, just posted a video of its unbreakable windows on social media, and went VIRAL.
EPIC TROLLING!
William Chen retweeted
important for all to note:
1 - venture returns are predicated on speed + scale
2 - reinventing a factory for speed + scale IS NOT "i need a factory to make x"
3 - venture equity is the only thing you can use to reinvent a factory
4 - reinvent it, then scale it with equity + debt
5 - scaling laws here far surpass most software companies especially on longer timelines, enormous cash flows and growth potential
For people who need pcb
made a tool for searching for PCB assy/fabs throughout the world: zackslab.com/atlas
if you're favorite pcb house isn't on here lmk :)
William Chen retweeted
the chinese are asking photonics PhDs to give training class to prisoners and make them do some of the assembling work, how are you going to move optics/photonics supply chain back to the US, calling a 4 pm meeting "in the evening"?
William Chen retweeted
American glass. American metal. American silicon. American batteries. American workers. American everything.
BlueChip is the first American computer in decades. Don't let anyone lie to you and say it can't be done.
Designed in California. Built in America.
Bc nobody uses AI except for startups in proportion to the 8 billion people in the world and that’s active usage not including ambient usage with agent swarms and sandboxes
Intelligence will and should become a commodity like water and for the most part electricity in the first world
Too cheap to meter: It is 0.025 cents (not dollars) per million tokens.
Look at the insane performance numbers for Kimi 3.1 on Vera Rubin
- 90$ worth (45M tokens * 2$ per million for input, assuming most tokens are input) of Kimi 3.1 tokens for 1$ of TCO at 100 tokens/sec interactivity. Insane margins. Lot of scope for players to cut prices.
It is true: The more you buy, the more you save 🙇♀️
- With theoretically same uplift for DS v4.1 Flash on Vera Rubin from Grace Blackwell (1600M per dollar TCO), there will be nearly 4000M tokens per dollar of TCO.
It is 0.025 cents (not dollars but cents) per million tokens.
(Grace Blackwell chart below for comparision. Then look at SemiAnalysis chart).
Explain me again how there is going to be 250B of inference GPU rental revenue next end year (25GW of inference ompute), and even higher GPU revenue year on year to justify 1T USD of per annum investment that will go on in perpetuity.
It is nearly 30$ per living person of GPU compute (8 billion population) and 120B tokens per living person (babies, kids, old people included) at 0.00025$ generation cost for a million tokens of DS v4.1 quality.
That is the level of consumption we need.
Either we have a glut or everyone has AI waifu, BF, dad, mom whatever they desire. I don't see everyone having a personal robot before 2027. I don't know about you.
Let me know if you disagree. These are relatively idealized scenarios. There would be lot of slack in the system and generation may be few tens of billions per living person, but even then it is a lot!!!
love this
PCBs made in America. Twice as fast as Shenzhen.
Today @CircuitHub is unveiling The Grid, our factory-scale robot in Massachusetts that builds circuit boards on demand.
No changeovers, no per-design setup. Prototypes and production batches, fully automated.
William Chen retweeted
Frontier labs making custom chips is not really as hard / as impressive as you might think.
1. They are primarily targeting inference not training. inference is easier
2. They are targeting transformers only. Think large MXU matmul units
3. They are writing all kernels in close to ISA with coding agents RLed for their ISA.
4. They aren’t making high level compilers for codegen
A frontier lab vending a true generic chip like a Nvidia GPU or Google TPU is MUCH harder than spinning up an in house inference program.
Still I think you will see many more inference chip companies as the cost of the software is just the cost of tokens.
William Chen retweeted
Western team have independently verified that DeepSeek's performance numbers of 38$ dollar revenue per GPU at DeepSeek's rocket bottom prices is true.
This is a momemnt when you update your priors. At this rate with just 1 GW, you can make a revenue of 150B per year.
Even with lot of slack it would be 100B per year. If you use half the compute to come online by 2027 end, you will have 25GW for inference with potentially 2.5T revenue at DeepSeek rates and 10T at USA lab rates (world GDP is 125 T, 10T is 8% of it)
You answer for your-self are we overbuilding!!!
vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64.
DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now matches DeepInfer's performance curves.
William Chen retweeted
Today I had in-depth conversations with optical, packaging, and system design engineers at Intel, NVIDIA, Marvell, Micron, Broadcom, and Ayar Labs.
My takeaway is that optics is just getting started. Don’t put a ceiling on how big it can get.
Investing is still hard. Conversations like these strengthen my conviction to keep going.
William Chen retweeted
If DeepSeek were American it would be on every magazine cover.
Think about it. A small lab with a fraction of the compute ships a frontier-class model under MIT. Then it open sources its production inference engine, its kernels, its file system. Publishes 155 trillion tokens a day and $38 of revenue per GPU-hour. Raises $15B anyway.
In America that is a national hero story. Senators would be posting about it. Every podcast would call it proof that open source made the country stronger.
Instead it got banned from US federal systems, called a security threat and accused of distillation, sometimes in the same week it gave away more of its production stack than OpenAI ever did.
The code is the same either way. The weights are the same. Only the flag changed the reaction.
I dont care about the flag. I care that I can download it, read it and run it on my own cards. By that measure DeepSeek did more for American developers this month than most American labs did all year.
William Chen retweeted
What the fuck are you even talking about? DeepSeek gave us a thousand times more than anyone else in this industry.
The weights, under MIT. Their production inference engine, under Apache 2.0, this week. Their attention kernels, their MoE communication library, their FP8 matmul library, their own distributed file system. The papers that explain how all of it works. And the component that makes your own deployment behave exactly like their API.
Now tell me what OpenAI and Anthropic gave us.
A $500 a month subscription. Limits that keep shrinking. Usage that drains faster than your plan says, until someone admits it was a bug. Models people swear got nerfed a month after launch. And a wave of account bans where nobody even tells you why.
Not 1 weight from Anthropic. Not 1 line of the code that serves ChatGPT.
Yes, the training data is not public, so call it open weights if you want. I do. But you picked the 1 lab that opened more of its real stack than anyone, and called it a lie.
Are you fucking out of your mind? You're the only one selling lies here.
DeepSeek sold you a lie.
Its new model, V4.1-Flash, is being sold as "open source".
It's not.
You can download it free, with no sign-up and use it in your business which sounds open source.
But you can't access the code they used to train it or the data they trained it on.
Without those, nobody can check how it was made or rebuild it and it’s over 700 billion parameters, so you can’t even run it on your laptop.
When you can build a business on something but can’t see how it was made, the right term is "open weights".
Call it what it is.
William Chen retweeted
Ben Horowitz explains why a16z doesn't invest in neoclouds after living through the dotcom bubble
"If you get into a pathological debt situation, that can wreck anybody. It's why, by the way, we're not invested in any of these neoclouds, for that reason. That whole thing scares me, having been through it."
"It can end up being a very dangerous arbitrage business where you can get upside down easily."
"You look at Loud Cloud just as an example, here's a simple thing that happened to us in 1999. Bandwidth was $1,500 a megabit. And you couldn't get it, it was very hard to get. And so you had to get a long-term contract, three years."
"By 2001 it was $50 a megabit. So if you were holding bandwidth, you were just dead."
"So that kind of thing is so out of the control of the entrepreneur. And so different than what we'd like to fund. That, you know, it just scared me to death."
William Chen retweeted
为什么我比较相信台积电对AI需求的判断？
因为这家公司经历过的半导体危机，实在太多了。
2001年互联网泡沫破裂，台积电第三季度产能利用率一度跌到41%。
2008年金融海啸，资本支出同比减少26%。
2023年半导体库存调整，资本支出又从363亿美元降到304亿美元。
但即使在行业低谷，台积电也没有停止先进制程的研发和升级。
经历了这么多轮周期，它应该比谁都清楚产能过剩的代价。
所以，当台积电计划2026年投入600–640亿美元，而且预计2026–2028年的资本支出还将显著高于过去三年时，我觉得这个信号值得认真研究。
不知道大家有没有注意到魏哲家10月15日法说会上的一段话？
台积电不只向客户了解需求，还会找客户的客户核实，甚至派自己的团队去美国，实地了解数据中心建设进度和电力供应。
云厂商的资本支出、融资能力，以及其他关键零组件的供应情况，也都是它评估需求的重要依据。
我觉得台积电还有一个经常被低估的能力，就是它多年积累的产业情报和需求分析体系。
从芯片设计、客户产品规划，到先进制程导入和未来量产安排，它掌握的产业信息，可能比大多数外部研究机构都要完整。
而且，这些信息还可以交叉验证。
以前做过多年供应链跟电子制造，我很清楚客户的Forecast和真正落地的订单是两回事。
尤其是半导体这种重资产行业，设备买回来、厂房建起来，客户就算推迟订单，折旧也不会跟着停下来。
所以，我不太相信台积电只是听客户讲了几个AI故事，就敢拿出这么多真金白银扩产。😂
如果一家经历过这么多轮半导体周期、拥有如此多一线产业信息的公司，经过反复验证后仍然决定大规模投资，我觉得这个判断很值得重视。
AI股票有没有泡沫？当然可能有。
但AI股票估值过高，和AI需求本身是不是泡沫，我觉得还是应该分开讨论。
比起市场每天冒出来的各种泡沫故事，我还是更愿意多看看产业链的实际变化。
这只是我的个人观察，也欢迎大家分享不同看法。👀
$TSM $NVDA
$TSM doesn’t build fabs on a whim. We’ve weathered semiconductor cycle after cycle, and TSMC is still standing.
On the ABF substrate side alone, Liwei is hearing that customers have already put down deposits to secure capacity through 2032.
Do you seriously think Taiwan’s suppliers would expand without customers putting money on the table first?
We in Taiwan have a stronger survival instinct than most. We know what’s at stake.
So stop fantasizing about the AI bubble bursting. As long as customers still have the cash and are willing to put it down to fight for capacity, this bubble isn’t bursting.
$INTC $MRVL $FORM $ASX $AEHR $AMAT
William Chen retweeted
菲尔兹奖得主、加州大学洛杉矶分校数学教授陶哲轩在最新访谈里罕见发火，直接点破了大模型刷榜背后一个没人收拾的烂摊子，
他说这场 AGI 竞赛正变得充满毒性，OpenAI 一口气倾倒出的 300 多篇论文，一夜之间端掉了年轻学者的初级研究课题。
他在访谈里算了一笔很无奈的账，面对这 300 多篇突如其来的 AI 论文，连他自己到现在也只顾得上看其中的 10 篇，
剩下的一大堆内容晦涩难懂，根本没人有精力仔细审完，甚至已经有一两篇因为被揪出错误而撤回。
外行看热闹的时候，习惯把 AI 批量证出数学定理当成技术突破来庆祝，觉得模型跑通了证明、发完新闻稿就算大功告成。
但陶哲轩直接戳破了这层泡沫，大厂为了多证几个定理、多发几篇公关稿、顺便测一把手里的新玩具，不惜把理解二字彻底抛在一边，这些 AI 只是在疯狂耗尽人类攒下的问题库，却没有给学科补回任何新的东西。
这就好比原本留给年轻学徒练手升级的新手村课题，被一台推土机连夜铲平了，
OpenAI 发完战报就觉得任务已经完成，也没有派人来协助清理，留给学术界的却是一堆没法拒绝的盲盒，里面好东西和垃圾混在一起，连比例是多少都没人摸得清，学者们反而得花海量时间去替 AI 验算排雷。
本质上，这给所有每天用 AI 写代码、做业务的人揭开了一个残酷的现实，
当生成内容的成本降到零，真正的瓶颈就瞬间变成了人类的审查带宽，不管是在顶尖科研还是日常工程开发里，只管倾倒未经验证的 AI 产出，不仅不是提效，反而是在给团队制造审核灾难。
不过看这段批评也得留个清醒，陶哲轩本人一直是数学界走在前列拥抱 AI 工具的学者，他一直在推动用 Lean 形式化语言辅助数学研究，
他真正反感的是大厂只管跑分公关、把人工核验成本甩锅给社区的做法，而不是否定 AI 参与科学探索本身。
当生成答案的成本变成零，不带理解的批量输出就不再是资产，而是全行业的审核负债。
知名博客主持人@dwarkesh_sp 最新的一期深度访谈，跟当世最顶尖的数学家陶哲轩，关于 AI 未来的深度对话，没有哗众取宠的爆点，没有套路化的叙事，只有顶尖大脑最真诚和最本质的思考，与其刷一下午的小视频，不如沉下心好好看完这场洗涤大脑和认知的对话
William Chen retweeted
Chamath says Anthropic should sacrifice valuation to raise at least $200 billion, because even that barely buys the compute needed to survive a correction
“Here’s my very specific thing about why I think they should price it to clear the market for $200 billion—and I’m specific about that number. They need, in my estimate, at least $200 billion. If you want to be super safe, a quarter-trillion gets the job done.
“What does the price need to be to attract that incremental dollar off the sidelines? Can they raise $80 billion in an IPO at $1.7, $1.8 or $1.9 trillion? I think so, handily. Can they raise $100 billion? Maybe it’s pushing it. At $150 billion, I think you need to be back down to $1.5 trillion.”
JCal: “You’re saying you want them to raise more money at a lower valuation so they have a war chest.”
Chamath: “$150 billion at $1.5 trillion is not enough. $200 billion barely gets them a few gigawatts. That is the bottom-line math if Anthropic is going to go through and survive an eventual correction.
“I think it’s much better for them to have $200 billion to $250 billion today than to try to ring the bell at $1.9 or $2 trillion for $80 billion, file a mixed shelf—which takes about six months—and hope the markets are in line.
“I just think there’s more risk in the future than there is today. Why take the risk? Price it to clear the market.”
Ben Horowitz reveals Amazon CEO Andy Jassy told him its AI compute is sold out beyond 2028, and says that shortage makes this fundamentally different from the dot-com bubble
“This shortage is so bad. You know, I was speaking with Andy Jassy, who runs Amazon, yesterday. And I said, yeah, like, we’re out of everything, you know, till 2028. He’s like, oh, it’s beyond 2028. Like, I don’t think we get stuff back, like, end of 2028, maybe. So everything else is sold out.”
“So he has the advantage of that, he’s got supply in a great shortage, which is extremely different than the internet bubble. The internet bubble, we didn’t have enough demand, because we had to build the internet, people had to get broadband, you know, you needed all this stuff people had to buy, you know, originally, they had to get their browser at price, all this kind of thing.”
“There weren’t enough consumers on the internet to absorb all the bandwidth that had gotten built out. And they had a massive amount of debt, not unlike we’re building today, although not as big. But the lack of demand killed it. We don’t really have a lack of demand in AI. So I think the dynamics will be different.”
“I do think, like, for sure, if you get into a pathological debt situation, that can wreck anybody. It’s why, by the way, we’re not invested in any of these neo-clouds either, for that reason.”