Software Engineer pre AI - TensorFold Contributor - Local LLM builds for DGX Sparks - Engineering Mentor to 1,200+ AI builders.
Bangkok, Thailand
Joined February 2026
- Tweets2.1K
- Following127
- Followers1.2K
- Likes625
Pinned Tweet
My DeepSeek-V4.1-Flash-1017 quant is NOW LIVE for TP=2 DGX Sparks.
Same 353 questions for all three packs:
new / Mia 2.9 / my old q28-v2
MMLU-Pro STEM: 93 / 91 / 90 (of 100)
LiveCodeBench: 14 / 11 / 11 (of 17)
MBPP+: 50 / 49 / 50 (of 60)
HumanEval+: 34 / 34 / 33 (of 40)
SciCode: 6 / 6 / 5 (of 10)
MATH-500: 36 / 36 / 36 (of 40)
MGSM: 40 / 40 / 40
BFCL tool calls: 39 / 39 / 39 (of 40)
Multi-step tool chains: 6/6, 6/6, 4/6
These are small samples, so I read it as "at least as good, in a smaller pack". It's not a blowout.
The gaps I care about are LiveCodeBench and the tool chains, which my old quant kept dropping. Calibrated it for coding, science and agent work.
Weights/Recipe are in the comments. Recipe has been updated with my current config but is mostly unchanged.
@MiaAI_lab I still don't know how you get them so good. I can't wait to use your Engine Aware Quant (EAQ) system when it's ready.
2.8 BPW, EXL3, 203.8 GB, running on two DGX Sparks with @ashxhart TensorFold.
- 1M context
- image input
- 2.1M-token cache total with NVMe offload.
As usual, if you have any issues, please let me know or submit a PR.
What will you throw at it first?
LocalRouter AI assistant. One DGX Spark. Full duplex.
This actually feels like Jarvis in a box! Built on @ashxhart TensorFold
๐ง Qwen 27B
๐ฃ๏ธ Drama 3 with streaming (Thanks @plotarmordev )
๐ผ๏ธ Qwen Image Turbo 2.1
๐ฌ MiniMax H3 video
๐ Whisper STT
~0.5s after I stop talking, it starts with its first word. Interrupt it anytime. Set the delay as you please for responses.
Normal models stay loaded and can be swapped as needed.
Want a video model up? Then Whisper and Drama will make space. Once the video is generated, start voice back up when the video is finished.
The goal is actually fully modular, so consider this a super early version of what it could be. MCP by default, but also ships with a simple UI you can extend as you please.
Thin by design, so everyone can make it their own.
What features do people want next? What about TP=2 or Mac Studio versions?
github.com/jayleaton/localroโฆ
More PR's incoming as I continue testing through the night
Reminder that everyone has outages at the best of times. @Railway is a good product, and what has happened is unfortunate.
Maybe @theo can build a better way to handle secrets/rotations without needing a full deployment.
Railway seems to be coming back online slowly.
This is a reminder that all systems can and will fail, and to build in redundancies in the event of something like this.
I don't personally think swapping to something like Vercel would matter here. They are just as likely to be hit by these kinds of attacks.
The beginning of the end. WTF is happening, everyone is getting hacked or had emergency security issues.
- Node
- PHP x2
- OpenCode
- Railway
- Gitlab
- Lite LLM
Itโs going to get much worse I think. Please be careful out there.
๐จ CRITICAL: GitLab flaw (CVSS 10.0) affects 12 years of releases.
CVE-2026-110264 could allow unauthenticated remote code execution on self-managed servers.
Read - thehackernews.com/2026/10/crโฆ
Clearly, @ThePrimeagen doesn't know about my 2 DGX Spark recipes for DeepSeek Flash on TensorFold.
I wonder if PornHub runs on PHP. Maybe some good will come out of bringing down over half the internet.
โผ๏ธ PHP UPDATE: The Dutch parliament, the Dutch national cyber agency and Finland's central government have pulled websites offline ahead of the PHP patch.
Finland's state IT centre Valtori is shutting down public-facing central government web services. The National Land Survey of Finland is already down.
In the Netherlands, the Tweede Kamer, the NCSC and the Dutch Data Protection Authority are offline. So are tech site Tweakers and NotuBiz, which runs council information for 100+ local governments.
The University of Vienna switched off its websites, webmail and Moodle. The University of Freiburg and Finland's Haaga-Helia have taken services down too.
Fur Affinity, Pinboard, explain xkcd and every wiki on Miraheze are dark as well.
Valtori says its shutdown is a precaution, not the result of an attack or breach.
PHP was created by Rasmus Lerdorf in 1994, it initially stood for Personal Home Page
PHP's fix is due at 14:00 UTC today.
Local ai is already doing about 30% of my work.
Once we have local models that perform on the same level as Opus 5.5 then all my work will be using local ai.
The only reasons I would consider paying for subs after that point comes down to 2 reasons.
1. Faster speed. If I can get 200-300 tokens per second at current regular prices then itโs worth it as itโs a multiplayer on output.
2. Complex math/science experiments. As ai solves more math and science problems I might use to experiment on new lossless compression or scaling improvements.
The future of ai improvements are going to be on speed and solving problems humans canโt. Opus 5.5 is already good enough for every task 90% of humans need done.
Welp. Hundreds of km away from my homelab and now one of my machines doesnโt want to boot.
Guess that means Il be working on 1 Spark solutions until I get home. ;)
Very good results for 1 DGX Spark! Incredibly smart and donโt know how @ViC305 did it.
This is the new level to set for 1 Spark.
It would be nice to see tool call improvements as that certainly will effect autonomous work if it gets stuck in a tool call loop often.
Another model I will have to try!
๐จ ๐ ๐ฌ ๐ญ.๐ฒ๐ฎ ๐๐ฃ๐ช ๐๐๐๐ฃ๐ฆ๐๐๐ ๐ค๐จ๐๐ก๐ง ๐๐จ๐ฆ๐ง ๐๐๐๐ง ๐ง๐ช๐ข ๐๐๐๐๐๐ฅ-๐ฃ๐ฅ๐๐๐๐ฆ๐๐ข๐ก, ๐ง๐ช๐ข-๐ฆ๐ฃ๐๐ฅ๐ ๐ค๐จ๐๐ก๐ง๐ฆ! ๐งช๐ฅ
And I have the REAL benchmark results to prove it! ๐คฏ
Remember my DeepSeek V4.1 Flash 1017 SAGE-EXL3 release from YESTERDAY?
๐ฆ 1.62 bpw routed experts
๐ป 99.61% code answer-token agreement
๐ฅ๏ธ ONE DGX Spark @nvidiaai
Well, I promised to go beyond KL divergence and token agreement and test ACTUAL coding, mathematics, reasoning and tool execution.
๐ง๐๐ ๐ฅ๐๐ฆ๐จ๐๐ง๐ฆ ๐๐ฅ๐ ๐๐ก!
โโโโโโโโโโโโโโโโโโโโ
๐ ๐๐๐๐ฃ๐ฆ๐๐๐ ๐ฉ๐ฐ.๐ญ ๐๐๐๐ฆ๐ ๐ญ๐ฌ๐ญ๐ณ โ ๐ก๐๐ก๐-๐ฆ๐จ๐๐ง๐ ๐ฆ๐๐ข๐ช๐๐ข๐ช๐ก
353 matched benchmark questions.
9 evaluation suites.
Strict pass@1 scoring.
Temperature 0.
๐ข ๐ ๐ฌ ๐ฆ๐๐๐-๐๐ซ๐๐ฏ โ ๐ญ.๐ฒ๐ฎ ๐๐ฃ๐ช
๐ฅ๏ธ 1ร DGX Spark
๐ ๐ต๐ฐ.๐ญ% โ ๐ฏ๐ฏ๐ฎ/๐ฏ๐ฑ๐ฏ ๐ฃ๐๐ฆ๐ฆ๐๐!
๐ต ๐๐ ๐ก๐๐ช โ ๐ฎ.๐ด ๐๐ฃ๐ช
๐ฅ๏ธ 2ร DGX Sparks
๐ 90.1% โ 318/353
๐ฃ ๐ ๐๐๐๐๐ โ ๐ฎ.๐ต ๐๐ฃ๐ช
๐ฅ๏ธ 2ร DGX Sparks
๐ 88.4% โ 312/353
๐'๐ ๐จ๐ฆ๐๐ก๐ ๐ฐ๐ฎโ๐ฐ๐ฐ% ๐๐๐ช๐๐ฅ ๐๐๐ง๐ฆ ๐ฃ๐๐ฅ ๐ฅ๐ข๐จ๐ง๐๐ ๐๐ซ๐ฃ๐๐ฅ๐ง ๐ฃ๐๐ฅ๐๐ ๐๐ง๐๐ฅ...
๐๐ก๐ ๐ฆ๐๐ข๐ฅ๐๐ก๐ ๐๐๐๐๐๐ฅ ๐ข๐ก ๐ข๐ก๐ ๐ฆ๐ฃ๐๐ฅ๐! ๐คฏ
โโโโโโโโโโโโโโโโโโโโ
๐ป ๐ง๐๐ ๐๐ข๐๐๐ก๐ ๐ฅ๐๐ฆ๐จ๐๐ง๐ฆ ๐๐ฅ๐ ๐๐ก๐๐ฅ๐๐๐๐๐๐!
Scores shown as SAGE / JJ / Mia:
๐ HumanEval+
39/40 vs 34/40 vs 34/40
๐ MBPP+
53/60 vs 50/60 vs 49/60
๐ LiveCodeBench
15/17 vs 14/17 vs 11/17
๐ง ๐ฅ๐๐๐ฆ๐ข๐ก๐๐ก๐ & ๐ ๐๐ง๐
๐ MMLU-Pro STEM
95/100 vs 93/100 vs 91/100
๐ MATH-500
40/40 vs 36/40 vs 36/40
๐๐๐ฉ๐ ๐ข๐จ๐ง๐ฅ๐๐๐๐ง ๐ช๐๐ก๐ฆ! ๐ฅ
I also tied both models on:
โ
MGSM: 40/40
โ
BFCL: 39/40
โ
SciCode first-step: 6/10
For full transparency, both JJ and Mia won the multi-step tool-chain test, scoring 6/6 against my 5/6.
๐ฆ๐ข ๐ง๐๐๐ง'๐ฆ ๐ฑ ๐ช๐๐ก๐ฆ, ๐ฏ ๐ง๐๐๐ฆ, ๐๐ก๐ ๐ญ ๐๐ข๐ฆ๐ฆ.
All while using a substantially lower expert bitrate!
โโโโโโโโโโโโโโโโโโโโ
๐ ๐๐๐ ๐จ๐ฃ๐๐๐ง๐: ๐ ๐ฌ ๐ฃ๐จ๐๐๐๐ ๐๐๐๐ ๐ฅ๐๐๐๐ฃ๐ ๐๐ฆ ๐ก๐ข๐ช ๐ฃ๐จ๐ฆ๐๐๐ก๐ ๐ฑ๐ฎ๐ฐ๐ ๐๐ข๐ก๐ง๐๐ซ๐ง!
YES. ON ONE DGX SPARK! ๐คฏ
With the latest W2 kernel, updated expert set and KV-cache configuration, I just completed an actual long-context retrieval test!
๐ ๐๐จ๐๐-๐๐๐ก๐๐ง๐ ๐ฃ๐ฅ๐ข๐ ๐ฃ๐ง ๐ง๐๐ฆ๐ง
๐ง 518,103 INPUT TOKENS!
๐ฏ Fact planted HALFWAY through the context
โ
THE MODEL FOUND THE FACT!
โฑ๏ธ Time to first token: 470 seconds (~7.8 minutes)
๐ Prefill: ~1,100 tok/s
โก Decode at that context depth: 25.6 tok/s
๐พ Minimum available memory: 2.19 GiB
๐ก๏ธ Memory safety floor: 0.5 GiB
And on SHORTER prompts?
โก ~36 TOK/S DECODE!
๐ง ~130.5 ms per speculative step
๐ฏ ~4.7 accepted tokens per step
๐ช๐'๐ฅ๐ ๐ง๐๐๐๐๐ก๐ ๐๐๐ข๐จ๐ง ๐ข๐ฉ๐๐ฅ ๐๐๐๐ ๐ ๐ ๐๐๐๐๐ข๐ก ๐๐ก๐ฃ๐จ๐ง ๐ง๐ข๐๐๐ก๐ฆ ๐ฃ๐ฅ๐ข๐๐๐ฆ๐ฆ๐๐ ๐ข๐ก ๐ ๐ฆ๐๐ก๐๐๐ ๐ฆ๐ฃ๐๐ฅ๐! ๐ฅ
To be clear, this was a successful full-length retrieval test, not a claim of perfect accuracy across every possible long-context task.
And the 7.8-minute prefill is a real trade-off at that depth.
But seeing a model this large, with this level of compression, successfully retrieve information buried inside 518K tokens on ONE 128GB machine is an exciting milestone for local AI!
โโโโโโโโโโโโโโโโโโโโ
โก ๐๐ก๐ ๐ง๐๐ ๐ข๐ก๐-๐ฆ๐ฃ๐๐ฅ๐ ๐ฃ๐๐ฅ๐๐ข๐ฅ๐ ๐๐ก๐๐ ๐๐๐๐ฃ๐ฆ ๐๐ ๐ฃ๐ฅ๐ข๐ฉ๐๐ก๐!
Separately, my latest REXL3 development measurements at a 256K context configuration:
โก 55.35 tok/s single-stream decode
๐ 2,045 tok/s peak prefill
โฑ๏ธ 509 ms TTFT on a 512-token prompt
Those REXL3 numbers come from a different engine and workload than the new public vLLM 518K-token test.
And here's something else I'm particularly excited about...
๐๐๐ ๐ญ๐ฑ,๐ฏ๐ฒ๐ฌ ๐ฅ๐ข๐จ๐ง๐๐ ๐๐ซ๐ฃ๐๐ฅ๐ง๐ฆ + ๐ฏ๐ด๐ฐ ๐๐ฅ๐๐๐ง ๐๐ซ๐ฃ๐๐ฅ๐ง๐ฆ ๐๐ฅ๐ ๐ญ๐ฌ๐ฌ% ๐๐ฃ๐จ-๐ฅ๐๐ฆ๐๐๐๐ก๐ง ๐๐ก ๐ ๐ฌ ๐ง๐๐ฆ๐ง๐๐ ๐ฅ๐๐ซ๐๐ฏ ๐๐ข๐ก๐๐๐๐จ๐ฅ๐๐ง๐๐ข๐ก!
ZERO streamed experts.
๐ข๐ก๐. ๐ญ๐ฎ๐ด๐๐. ๐๐๐ซ. ๐ฆ๐ฃ๐๐ฅ๐. ๐
@mr_r0b0t and I are continuing to refine REXL3 before its release. More on that soon! ๐
โโโโโโโโโโโโโโโโโโโโ
๐ ๐ง๐๐๐ฆ ๐๐ฆ ๐ช๐๐ฌ ๐ ๐๐จ๐๐๐ ๐๐ข๐ฅ ๐๐ข๐๐๐ ๐๐.
Just YESTERDAY, I shared my goal of bringing extremely aggressive, high-quality quantization to people who don't have access to massive compute clusters.
With the support of @AWERXAI, I've been working tirelessly to push that vision forward.
And now we're seeing a 1.62 bpw expert quant outperform higher-bitrate alternatives on actual tasks, not just token-level fidelity measurements.
This doesn't establish mathematical losslessness or superiority on every possible workload. These are matched subsets of nine benchmark suites, not full public leaderboard scores.
But it demonstrates something IMPORTANT:
๐ ๐ข๐ฅ๐ ๐๐๐ง๐ฆ ๐๐ข๐ก'๐ง ๐๐จ๐ง๐ข๐ ๐๐ง๐๐๐๐๐๐ฌ ๐ ๐๐๐ก ๐๐๐ง๐ง๐๐ฅ ๐ฅ๐๐ฆ๐จ๐๐ง๐ฆ.
And more hardware doesn't automatically mean you need it!
Huge respect to @jayleaton for his contributions and for giving the community more ways to run these incredible models.
And thank you @AWERXAI for supporting this work. ๐
๐ ๐ ๐ฌ ๐ฆ๐๐๐-๐๐ซ๐๐ฏ ๐ค๐จ๐๐ก๐ง
huggingface.co/vcruz305/DSV4โฆ
๐ ๏ธ ๐ข๐ก๐-๐ฆ๐ฃ๐๐ฅ๐ ๐๐๐๐ ๐ฅ๐๐๐๐ฃ๐
github.com/vcruz305/DSV4.1-Fโฆ
๐๐ฅ๐ข๐ก๐ง๐๐๐ฅ ๐ ๐ข๐๐๐๐ฆ. ๐๐ข๐๐๐ ๐๐๐ฅ๐๐ช๐๐ฅ๐. ๐ฅ๐๐๐ ๐ฅ๐๐ฆ๐จ๐๐ง๐ฆ.
๐๐๐ง'๐ฆ ๐๐๐๐ฃ ๐ฃ๐จ๐ฆ๐๐๐ก๐! ๐๐ฅ
TP=4 this is your model.
@deepseek_ai -V4.1-Flash-1017 is live on 4ร GB10 with TensorFold TP4.
โข Our EXL3 4.0 bpw quant of the new 1017 weights: open, MIT
โข 136 tok/s code at 1 stream, 183 tok/s at 4 streams
โข 2,679 tok/s prefill at 128K
โข 1M context, vision, bit-exact drafting
huggingface.co/bertholomus/Dโฆ
Crazy that I can sit on a beach in the Phuket islands while my AI computers in my home office can run all my work.
@bot does all my email/calendar and triage for any product outages or help tickets.
DGX Sparks are now self improving the recipe I released last night with my new quant.
My Mac Studio is working on some Mac tensorfold stuff.
My Linux homelab is running all my usual work.
I just use my own fork of T3 for agents and Grokbot app and I have merged over 100 prโs in the last 36 hours across 5 projects.
Everyone is doing quants now and thatโs a good thing.
I want everyone to push different boundaries so we can make the whole industry better.
I ran my quant all night and it found 3 potential opportunities to improve itself so now Claude is verifying what it found.
I push these models because I actively use them for real engineering work.
A large amount of @lockpickapp was built with DeepSeek because Claude always flagged as a security risk even though itโs a defensive product.
I even had to write some code myself and use obliterated models as even GLM & DeepSeek refused some of the really risky stuff.
Using platforms like Lovable, Bolt, Replit, and any platform in a can solution is just like when our parents wasted money on TimeShare.
Sure, they look & sound like a solid idea, but the whole design is just to make money and not actually solve anyone's direct problems.
You will spend all your money on a solution that doesn't work, and it will be impossible to leave without rebuilding most of the systems.
Just like TimeShare, if you just use the right solution when you actually need it, you will likely spend less money, be better located and move through everything much quicker.
I predict these "builder" platforms will be gone in 6-12 months.
Claude will probably ship a templates store and "Claude hosting" directly before long.
Early numbers on my NEW DeekSeek-V4.1-Flash-1017 quant.
- EXL3 & TP=2
- 2.80BPW
- 203GB
- 2.4M cache (1M per chat, NVMe offload once full)
Compared against the full 1017 model
- Code 95.17%
- Science 94.93%
- Tool Call 87.64%
Performance
- Prompt Processing 2631 (unchanged)
- Chat is mixed. 5% faster for short responses and 5% slower for longer ones.
My goal was to make it smarter where it matters. The tool call in the previous version was already good enough, so I wanted to match that while focusing as much as possible on engineering knowledge.
On average, it will be faster for things like chained tool calls and agentic workflows but slightly slower for longer thinking or response sessions.
It's running full benchmarks now and will upload after that.
