Local LLM V100 96G & enjoy Omarchy; Tday https://xitter.cf/t.co/P6K9F7LvIO ; CODELF (Github star 14k); MIHTool (Mentioned in Google I/O'13)
Los Angeles, CA
Joined January 2008
- Tweets4.6K
- Following387
- Followers2.4K
- Likes1.1K
Pinned Tweet
We just hit 300 members today—woke up to 100+ new people joining overnight. Come join these top DIY hackers using extreme optimization to make old hardware run Qwen3.8-Flash and GLM5.3-Flash. 🚀xitter.cf/i/chat/group_join/g209…
Just a bit of insight: people are eager to build their own rigs for local LLMs, the same way we were obsessed with building gaming PCs 30 years ago. xitter.cf/i/chat/group_join/g209…
Qwen3.8-Flash-Next GSQ-RCO-IQ3_S vs UD Q4_K_XL—GSQ-RCO-IQ3_S(left) is 1.3x decode faster but both ran in 32 minutes, but UD Q4_K_XL(right) clearly wins on detail.
4× V100 96GB VRAM test on ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S: Strata is far ahead of llama.cpp+DFlash—9.05× prefill, 1.67× decode. Can’t wait for Strata + DFlash + this model. And the quant quality is f***ing good—on par with Q6 so far. Gonna run more tasks on it.
GSQ-RCO quietly dropped a version (IQ3_S) of Qwen3.8-Next-Flash that exceeds original BF16. Needs around 57 total GB (VRAM+RAM).
Engine training in progress for you to run on consumer hardware ;)
huggingface.co/ISTA-DASLab/Q…
github.com/Niko1221/Strata
OMG, 4000 TPS prefill. I’ve never seen such speed on my V100 rig.😍
GSQ-RCO quietly dropped a version (IQ3_S) of Qwen3.8-Next-Flash that exceeds original BF16. Needs around 57 total GB (VRAM+RAM).
Engine training in progress for you to run on consumer hardware ;)
huggingface.co/ISTA-DASLab/Q…
github.com/Niko1221/Strata
Bad news, NVIDIA CMP 170HX price curve is insane: $200 last year, $1,500 last month, now over $3,000—still less than half an RTX 5090. Specs are wild: A100-class performance, 64–80GB VRAM, bandwidth close to an RTX 5090, SM80 compute, and CUDA 13 support. If you don’t want to tinker with V100s, an unlocked 170HX is the no-brainer combo of value and convenience. Good news, supply is still plentiful.
🚀 Local inference engines worth following: @coldniko (Strata/Qwen3.8-Flash), @MiaAI_lab (DGX Spark), @inco_ai (Splash+DFlash/Mac), @Andy_ShuoYang (FreeToken/NVFP4), @PeasantSmith (Maya/GLM5.3-Flash).
Must have @coldniko 😎🔥
⚡️ Along with this version, we've also released our DFlash 2 drafter for Qwen3.8-Flash-Next!
huggingface.co/incoai/Qwen3.…
In China, a Mac Studio M5 Ultra with 512GB costs about the same as BYD’s hottest new SUV or sedan. Imagine that: the price of a car almost any family can afford now buys you the ability to deploy world-class SOTA models at home. A few years ago, that picture would have been unbelievable.
Bad news: Only supports SM100+(B100 / B200/ RTX5090/ RTX5080) and full GPU offload.
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
We just hit 300 members today—woke up to 100+ new people joining overnight. Come join these top DIY hackers using extreme optimization to make old hardware run Qwen3.8-Flash and GLM5.3-Flash. 🚀xitter.cf/i/chat/group_join/g209…
This great man deserves to be followed—come on, guys.
Our largest model yet.
763B parameters, compressed with GSQ-RCO and co-optimized with Strata (by @coldniko) for local inference.
On Hugging Face this Friday.
Much more is coming. Follow us for future releases.
No need to scramble actually—DDR3 is more than enough, and there’s plenty to go around. tomshardware.com/video-games…
Last year I never thought I could cancel all my online SOTA subscriptions. This year, Strata + Qwen made it happen. Want to do it with a $1000~1,500 DIY build? Factor in local power costs. People ask about draw—mine’s under $1/day when system on, since it’s not always at full load. What’s electricity like where you are?
🤖 Made with AI
Tempting me to upgrade to 256GB RAM 😭 Good news: even IQ1 hits 88% top-1—already beating Qwen3.8-Flash Q4—and IQ2 reaches 94%, close to Qwen3.8-flash Q6. Doesn’t feel out of practice anymore. Bad news: you need 128GB of RAM + VRAM just to start, and there’s no Strata-style optimization for older GPUs. The real-world barrier is still high.
DeepSeek V4.1-Flash-1017 can now run locally! 🐳
We made 1-bit retain ~88.2% accuracy while shrinking size from 508GB to 250GB (−52%).
Run on a 128GB+ RAM device.
V4.1 Flash 1017 is the strongest open model to date
Guide: unsloth.ai/docs/models/deeps…
GGUF: huggingface.co/unsloth/DeepS…
