Local LLM V100 96G & enjoy Omarchy; Tday https://xitter.cf/t.co/P6K9F7LvIO ; CODELF (Github star 14k); MIHTool (Mentioned in Google I/O'13)
Los Angeles, CA
Joined January 2008
- Tweets4.6K
- Following387
- Followers2.5K
- Likes1.1K
Pinned Tweet
We just hit 300 members today—woke up to 100+ new people joining overnight. Come join these top DIY hackers using extreme optimization to make old hardware run Qwen3.8-Flash and GLM5.3-Flash. 🚀xitter.cf/i/chat/group_join/g209…
Just a bit of insight: people are eager to build their own rigs for local LLMs, the same way we were obsessed with building gaming PCs 30 years ago. xitter.cf/i/chat/group_join/g209…
DeepSeek V4.1 Flash “cheated” by checking Qwen’s previous work from memory, then ate up the 70 minutes I’d planned to sleep—plus $3. But I forgive you. The quality won me over. Take my knees—SOTA King! 🙇
4× V100 96GB VRAM test on ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S: Strata is far ahead of llama.cpp+DFlash—9.05× prefill, 1.67× decode. Can’t wait for Strata + DFlash + this model. And the quant quality is f***ing good—on par with Q6 so far. Gonna run more tasks on it.
GSQ-RCO quietly dropped a version (IQ3_S) of Qwen3.8-Next-Flash that exceeds original BF16. Needs around 57 total GB (VRAM+RAM).
Engine training in progress for you to run on consumer hardware ;)
huggingface.co/ISTA-DASLab/Q…
github.com/Niko1221/Strata
OMG, 4000 TPS prefill. I’ve never seen such speed on my V100 rig.😍
GSQ-RCO quietly dropped a version (IQ3_S) of Qwen3.8-Next-Flash that exceeds original BF16. Needs around 57 total GB (VRAM+RAM).
Engine training in progress for you to run on consumer hardware ;)
huggingface.co/ISTA-DASLab/Q…
github.com/Niko1221/Strata
Bad news, NVIDIA CMP 170HX price curve is insane: $200 last year, $1,500 last month, now over $3,000—still less than half an RTX 5090. Specs are wild: A100-class performance, 64–80GB VRAM, bandwidth close to an RTX 5090, SM80 compute, and CUDA 13 support. If you don’t want to tinker with V100s, an unlocked 170HX is the no-brainer combo of value and convenience. Good news, supply is still plentiful.
🚀 Local inference engines worth following: @coldniko (Strata/Qwen3.8-Flash), @MiaAI_lab (DGX Spark), @inco_ai (Splash+DFlash/Mac), @Andy_ShuoYang (FreeToken/NVFP4), @PeasantSmith (Maya/GLM5.3-Flash).
Must have @coldniko 😎🔥
⚡️ Along with this version, we've also released our DFlash 2 drafter for Qwen3.8-Flash-Next!
huggingface.co/incoai/Qwen3.…
In China, a Mac Studio M5 Ultra with 512GB costs about the same as BYD’s hottest new SUV or sedan. Imagine that: the price of a car almost any family can afford now buys you the ability to deploy world-class SOTA models at home. A few years ago, that picture would have been unbelievable.
Bad news: Only supports SM100+(B100 / B200/ RTX5090/ RTX5080) and full GPU offload.
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
We just hit 300 members today—woke up to 100+ new people joining overnight. Come join these top DIY hackers using extreme optimization to make old hardware run Qwen3.8-Flash and GLM5.3-Flash. 🚀xitter.cf/i/chat/group_join/g209…
This great man deserves to be followed—come on, guys.
Our largest model yet.
763B parameters, compressed with GSQ-RCO and co-optimized with Strata (by @coldniko) for local inference.
On Hugging Face this Friday.
Much more is coming. Follow us for future releases.
No need to scramble actually—DDR3 is more than enough, and there’s plenty to go around. tomshardware.com/video-games…
