0xCS

"It is impossible to design a system so perfect that no one needs to be good." - TS Eliot The quote attributed to T.S. Eliot highlights the philosophical and ethical understanding that no matter how well-designed a system is, it cannot entirely compensate for the moral character and intentions of the individuals interacting with it. In the context of AI, this quote suggests that while we can strive to create advanced and robust AI systems with many safeguards, these systems cannot fully preempt or prevent all potential misuse by individuals who are determined to behave unethically. AI systems rely on human users and developers to use them responsibly and ethically. This is why the role of human oversight, ethical guidelines, and continuous monitoring is crucial in the development and deployment of AI technologies. It's a reminder that technology alone cannot solve all problems related to human behavior and that ethical use is a shared responsibility.

Really useful engineering write-up! What stands out most is the practical, experience based approach. Real world measurements, reproducible checks, rollback procedures, and lessons learned from actually running the hardware. Not just another installation guide, but genuine hands on engineering experience shared with the community. Exactly the kind of documentation the DGX Spark community needs. Great work!

Canonical has now pushed fixes for CVE-2026-98432 into Ubuntu’s proposed pocket for several standard kernels. Important for DGX Spark users: this does not mean the Spark is fixed yet. My 4× DGX Spark cluster is running: 7.0.0-1024-nvidia Package: linux-nvidia-7.0 Canonical still lists linux-nvidia-7.0 on Ubuntu 24.04 as Vulnerable, with no fixed Spark/NVIDIA kernel version published yet. So for now I’m keeping the mitigations in place: • net.mptcp.enabled = 0 • net.ipv4.tcp_migrate_req = 0 • nftables drops IPv4 TCP packets carrying IP options • mitigation verified persistent after reboot on all 4 nodes The good news: Canonical’s generic kernel fixes are already going through qualification. The important part: do not remove the DGX Spark mitigation until linux-nvidia-7.0 is explicitly marked fixed. ubuntu.com/blog/fixes-availa…

Very interesting results. For me the most valuable part here is not really the ~100 tok/s single request TP8 number, but what happens once long cold prefills and concurrency are introduced. At 100K code context, C4 → C8 only moves actual batch output throughput from 34.02 to 36.05 tok/s roughly +6% while median TTFT increases from 63.84s to 113.65s. At 200K it becomes even clearer: 16.28 → 16.82 tok/s, while TTFT goes from 144s to ~260s. That suggests the important scaling limit for an interactive 8× Spark deployment is no longer simply available KV capacity or decode compute. Prefill interference/admission scheduling becomes a first class problem. Having spare request slots clearly does not mean having spare latency capacity. The vLLM TP8 numbers posted later in the thread are also very interesting: ~125 tok/s code / ~111 tok/s prose single stream and ~471–479 tok/s at eight requests. I would be careful with a direct comparison, though. That implementation is using a heavily optimized TileLang/vLLM stack, different RoCE/collective work, Engram optimizations, and the posted test uses max_tokens=256 versus 1,024 here. So this is not yet enough evidence to reduce the difference simply to “vLLM vs SGLang.” Excellent work collecting the TTFT and inter token gap data. Those numbers are much more useful for deciding how to operate an 8× Spark service than a single headline tok/s benchmark. forums.developer.nvidia.com/…

I’m visiting Greece, and I’m discovering that it’s much more than islands and beaches. The mountains of Metsovo and the Pindus are incredible deep valleys, dramatic cliffs, endless forests and clouds drifting between the peaks. 🍂

Congratulations!

Fingerprint database. 😂

One month later, I’m even happier with the decision to invest in NVIDIA hardware and build the 4× DGX Spark cluster instead of moving to a larger Mac. Since then, what has impressed me most isn’t only the hardware performance. It’s the velocity of the ecosystem around it. New models, new quantization methods, distributed inference, post-training, multi-node recipes, networking optimizations, speculative decoding, better runtimes almost every week there is something new to test. And seeing companies like Benchling working with NVIDIA to post train open models such as Nemotron 3.5 Super VL on proprietary scientific data reinforces exactly why I made that choice. The direction is becoming clearer: The future is not only about having enough memory to run a large model. It is about having a platform where you can run it, modify it, train it, distribute it across multiple machines, optimize it, and build specialized intelligence around your own data. I loved my M2 Ultra, and I still use Apple Silicon with my M5 Max. But for serious AI experimentation, I’m very happy I invested in NVIDIA. 4× DGX Spark + M5 Max was the right decision for me.

This is exactly why Apple is being left behind in the serious AI platform race. Benchling took NVIDIA’s open weight Nemotron 3.5 Super VL, post trained it on proprietary scientific protocol data, and produced specialized intelligence that outperformed comparable closed models in troubleshooting and optimizing laboratory experiments at remarkably low cost. That is the difference. NVIDIA isn’t just selling AI hardware or giving developers access to a model. It’s building an ecosystem where companies can take an open model, train it on their own proprietary knowledge, deploy it on their own infrastructure, and keep improving it over time. Apple has excellent silicon and MLX. But NVIDIA is building the entire AI factory.

$23,799 for a Mac Studio. Apple built the Mac Studio that makes a DGX Spark look reasonably priced. 🍎💸

Liquid AI may not become the next OpenAI. It could become something more quietly fundamental: the intelligence layer embedded everywhere inside devices, vehicles, robots, sensors, and software doing for compact AI what Arm did for computing infrastructure.

If the rumored 2.5-3.5T parameter size for GLM-5.5 turns out to be real, its launch will define whether my DGX Spark cluster expands from TP4 to TP6. Current cluster: 4× DGX Spark = 512 GB unified memory Potential upgrade: 6× DGX Spark = 768 GB unified memory My decision rule is straightforward: If GLM-5.5 cannot run comfortably on TP4 at a quantization level I consider acceptable, but can run properly within ~700 GB on TP6, I’m adding two more Sparks. That means enough memory not only for the weights, but also for KV cache, quantization metadata, runtime buffers, routing overhead, and a useful context length. The raw numbers explain why: 2.5T @ 4-bit ≈ 1.25 TB 2.5T @ 2-bit ≈ 625 GB 3.0T @ 2-bit ≈ 750 GB 3.5T @ 2-bit ≈ 875 GB So 2.5T at ~2-bit is exactly the kind of model that could trigger TP6: ~625 GB of raw weights against 768 GB of aggregate memory. But there’s also a hard limit to the plan: If the practical footprint is already ~750 GB+ before runtime headroom, I won’t add two Sparks just to force it into TP6. So for me, GLM-5.5 has a very concrete outcome: Fits TP4 → stay at 4 Sparks. Doesn’t fit TP4, but fits TP6 properly → buy 2 more Sparks. Doesn’t realistically fit TP6 → wait for a smaller/Flash variant or a better quantization path. GLM-5.5 could literally be the model that decides whether this cluster becomes 6× DGX Spark