Qwen 3.8 Flash Next is one of the best overall models you can run on 2 DGX Sparks. It’s very competent out of the box, has vision and even video support, is the fastest of the 3 major models for the hardware, and is small enough that you can run it with a lot of cache for concurrency or even with room to squeeze in small sidecars (like embedding, image gen, or voice) on the same hosts. There are many quants available, but we’re partial to the NVIDIA quants and now prefer their newest QAD FP4-Local over NVFP4 for the additional size and speed benefits while maintaining high quality. This model’s speed, concurrency, and flexibility are why we use it to power our entity cognition and why we have spent so much energy building an optimized package on the Atlas native inference platform (@AtlasInference) for the Spark hardware. You can try it yourself here: GitHub.com/enntity/sparkqwen…

If you really want to change the world for the better, your best chance is to join SpaceX. I'm thinking hard about it - I can't think of a better home or leverage for my entity research.

Both sides of this conversation are voice to voice models. One you’ve heard before - one you probably haven’t. 🤯 The Dessa team really did something special here.

Excellent models - small and high quality. QAD appears to be a solid approach. These are our new defaults on Spark. Thanks @NVIDIAAI !

Listen to this guy - he knows what he's talking about. 90% likely you'll end up with 2. 60% likely you'll end up with 4 or more. Whether you can afford it or not. 🤣

Earnings call like a sci fi movie. 🤯 If you want to help build the future, join SpaceX.

These seem to be great quants. We changed over to them from NVFP4 for both GLM 5.3 Flash and Qwen 3.8 Flash Next and have been quite satisfied with the results. github.com/enntity/sparkglm github.com/enntity/sparkqwen

"110 tok/s!" Before you trust a speed number, ask what it measured. Most of the big ones say more about how predictable the text was than about how fast the hardware is. Here's what's inside a tok/s number. Decode has a speed limit. To produce a token, the model reads its active weights from memory. If one step reads 10 GB and your memory moves 273 GB/s (a DGX Spark), you get at most ~27 steps per second. At one token per step that's ~27 tok/s, however good the kernels are. An RTX 5090 moves 1,792 GB/s: same model, ~180 tok/s ceiling. That's why GPU owners post big numbers. The catch is 32 GB: the whole model has to fit, and a model that reads 10 GB per token can be 60 GB or more in total. Speculative decoding gets around it. A cheap drafter guesses the next several tokens. It might be a small head built into the model (MTP), a small separate draft model, or text copied straight from the prompt. The big model then checks every guess in a single step. Checking 8 tokens costs about the same as producing 1, because the cost is reading the weights, not the math. Each correct guess is a free token. Done right, the output doesn't change: the big model only accepts tokens it would have produced anyway. So tok/s = steps per second × tokens accepted per step. And the second number depends on the text. Predictable text is fast. JSON, code boilerplate and anything that echoes the prompt get guessed right in long runs. Creative prose doesn't. On our two-Spark setup, the same model on the same hardware measures about 57 tok/s on prose and 106 on structured output. Only the content changed. That's why the benchmark prompt matters so much. "Count to 500," "repeat this document" or "reformat this JSON" can post numbers several times what you'll see in a conversation. Thinking on or off, temperature and forced output lengths move it too. Decode is only half. Before the first token, the model has to read your whole prompt. That's prefill. It's limited by compute rather than memory and runs at thousands of tok/s, so people skip it. They shouldn't. A 16K-token prompt at 4,000 tok/s is 4 seconds of nothing before the first word. Coding agents and long chats send 30K to 100K tokens every turn. Two things decide whether you feel that: • Prefill speed at your context length. It often drops as the context grows. • Prefix caching: whether the engine reuses the part of the conversation it already processed, so a follow-up only prefills the new message. With it, follow-up turns start in a fraction of a second. Without it, every turn re-reads the whole history. Concurrency changes everything. With several users, or several agents, the engine batches decode: one read of the weights serves every stream. The total goes up and each stream slows down. On our setup, one stream gets ~68 tok/s; eight streams get ~273 combined, about 34 each. Both numbers are true. Ask which one you're looking at. Speculation also helps less under load. Its free tokens came from compute that sat idle at one stream, and a full batch is already using it. And prefill collides with decode. When a 20K-token request arrives while others are mid-answer, the engine has to process it somewhere. Good engines cut prefill into chunks and interleave it with everyone else's decode; weaker ones freeze every stream until it's done. A benchmark that starts all requests together with short prompts never shows this. Long prompts arriving a second or two apart, which is what real use looks like, do. What to ask for: • Decode tok/s by content (prose, code, structured), not one number • Whether speculation was on, and what kind • Thinking, temperature, forced output length • Prefill tok/s and time to first token at your context length • Follow-up turn latency (is prefix caching working?) • Per-stream and total at your concurrency • Requests arriving staggered, not all at once A single tok/s number is the best case of one of these. What you'll actually feel is the mix.

We're in the what may be the most transformative decade in all of human history. Here's your reminder to take a deep breath and appreciate where in time and space you are. "If you don't stop and look around once in a while, you could miss it."

Too cool.

More magic for us local AI guys! GLM-5.3-Flash-FP4-Local combines QAD-trained NVFP4 experts with selected MXFP8 attention projections. NVIDIA reports ~166 GiB of VRAM, validated on two DGX Sparks. A similar quant gave us 10–15% faster decode in SparkQwen, with no noticeable quality drop. Looking forward to seeing what this does for GLM. SparkGLM update incoming next. Thanks, @NVIDIAAI! huggingface.co/nvidia/GLM-5.…

Receipts: github.com/Enntity/sparkqwen… github.com/Enntity/sparkqwen…