PhD in AI, in progress. I put discarded enterprise hardware back to work for local AI. Open weights. Real boxes. Measured numbers.
USA
Joined October 2021
- Tweets19.1K
- Following370
- Followers1.3K
- Likes14K
Steve | Local AI Hardware retweeted
Our partner ecosystem showed out with their NVIDIA Vera Rubin NVL72 racks on the showroom floor at #OCPSummit26!
Thank you to our partners for the ongoing collaboration. 🙌
Ouch. But it will sell out.. w/Tax? About $26K
Two of them for 1 TB? About $52K roughly, or just a little shy of a Tesla Model 3 Performance.
We are not in Kansas anymore, Dorothy.
We have seen prices escalate rapidly over the past 3 months. We need to get a sense of " years " in our heads.
Steve | Local AI Hardware retweeted
Nvidia gave some DGX Spark love today. It profiled Link 3.0 Flash (128B model) and improved decode.
37.8 → 53.9 tps.
No hardware upgrade. Thanks, Nvidia!
It stacked three software optimizations
⚙️ Humming MXFP4 MoE kernels
🧮 FP8 LM-head quantization
🚀 built-in MTP/NEXTN speculative decoding
On the full 1,319-question GSM8K evaluation, it improved
Decode
37.81 → 53.89 tps 42.5% speedup.
Mean latency
15.21s → 11.49s
Accuracy
96.89% → 96.82%
So basically the same measured accuracy while getting a lot more decode from the same hardware.
And the synthetic test gets even crazier:
36.09 → 102.88 tps
Link in ALT
ALT https://developer.nvidia.com/blog/3-ways-to-boost-moe-inference-on-nvidia-dgx-spark/
I spent the afternoon developing a dashboard for the DGX environment. A number of changes and updates, but right now it is in alpha form.
Spark Glass - Fleet dashboard
Right now you see mock data, but as soon as I get GLM flash up with TensorFold (@ashxhart @MiaAI_lab ) I am going to pull data from the engine into this as well.
This is my first run at developing a “single pane of glass” for the DGX environment.
More to come.
I have been watching the new feeds on the RTX platforms being crippled, and many are unlocking nearly double the tensor cores.
So I had a thought: could this also be true for the GB10?
I would not know where to start, but just the possibility is intriguing.
Steve | Local AI Hardware retweeted
🚨 ELON MUSK JUST ISSUED A TERRIFYING WARNING: 200+ AI CLOUD COMPANIES COULD FACE A BRUTAL WIPEOUT.
Musk says debt markets have almost completely stopped lending to companies that depend heavily on AI data centers for revenue.
He warns that most of the 200+ neocloud companies may not survive the coming wave of failures and consolidation.
And the situation could get much worse in 2027.
According to an AI infrastructure industry participant, major delays are building across data center projects, while many neocloud companies are already poorly capitalized.
This could leave billions of dollars worth of GPUs and computing infrastructure sitting idle or generating too little revenue to cover their costs.
Meanwhile, hyperscalers could prioritize their own computing needs, making it even harder for smaller AI companies to compete.
Musk revealed that SpaceX secured $40 BILLION in GPU-related debt financing, but it has a major advantage: revenue from Starlink and rocket launches.
Most smaller AI companies don't have that protection.
And remember, these companies have borrowed enormous amounts of money expecting AI demand to keep growing.
If their infrastructure fails to generate enough revenue, they could struggle to repay lenders, forcing asset sales, bankruptcies and massive losses for investors.
Musk believes the industry is heading toward a brutal consolidation, with only a small number of neoclouds surviving.
The AI boom might continue, but 2027 could be the year when many of the companies financing it get wiped out.
Holy...
JUST IN: Microsoft $MSFT offers buyouts to 60,000 employees as it cuts costs amid AI adoption.
This week's Artificial Intelligence blog entry is up. Follow me for weekly content on AI and Technology History.
thedigitalarchives.wordpress…
Good morning, Local AI Community
I just came back from a customer meeting with one of the largest Cisco customers globally, and the discussion we were having about hardware was, well, even worse than what I anticipated.
I am putting together an article to be released later today. When you get insight from the world's largest manufacturers on what their forecast and plan are for the next 12 months, it gives you a VERY clear picture of the impact we will feel in the Local AI space.
Steve | Local AI Hardware retweeted
Huge inventory of RTX 5090 GPUs appears in a Chinese factory, where technicians remove their GPU chips and VRAM to deploy them on custom boards. 🔗 wccf.tech/1lfi9
Steve | Local AI Hardware retweeted
GLM 5.5/OMNI will be a shift from Moore's Law to RSI
(Recursive Self-Improvement) in
Al is officially here.
Key takeaways from Tang Jie (Founder, Zhipu Al) on what's driving the next era of foundation models:
Execution > Chat: Value is moving from conversational Ul to long-horizon task completion.
Scaling Bias: Small-scale experiment conclusions don't linearly translate to trillion-parameter systems.
The RSI Loop: Models creating environments →
evaluating
rewards • training themselves.
Full credit to @NFT_Chen who has been the absolute master of breaking news!!!
🤖 Made with AI
It has been 7 days since the last article, and the impact of the memory shortage is starting to accelerate in Q4
As said, we are only at the tip of the ice berg. If the past 2 months have taught you nothing, the next 24 will humble many.
Be glad you were early, and NEVER listen to any post that says “sell your xxxx” ever.
🚨 PC HARDWARE MARKET UPDATE 🚨
We just received another industry notice confirming DDR5 memory price increases of 10–15% effective this week.
Even existing open orders are being canceled by this specific vendor and must be resubmitted at higher pricing.
GPU shortages, rising RAM costs, and tightening allocations continue to impact PC pricing.
If you're planning a build, waiting may cost you more.
Steve | Local AI Hardware retweeted
The Department of Homeland Security is requiring all civilian federal agencies to remove DeepSeek products and 'any third-party services developed using DeepSeek implementations,' and to stop buying them.
Follow my weekly blog covering the latest in Artificial Intelligence as well as the history of technology that got us here.
thedigitalarchives.wordpress…
