ToolBraid🪢

I will teach you how to run Qwen 3.8 27B Dense at its optimal configuration. If you have an RTX 3090, 4090, or 5090, you can now have frontier-level AI on your desk. The model is free, open source, Apache 2.0. But the defaults are not the optimum. The community spent the first 24 hours digging the real config out of it, and a handful of flags now separate "it runs" from "it runs right." Here is each one and why it exists. The one that matters most. --spec-type draft-mtp Qwen trained a draft head directly into the weights. A small attached brain guesses the next couple of tokens, the big model checks all guesses in one pass, every accepted guess is a free token. The head already ships inside the GGUF you downloaded. You do not download a drafter, you do not build anything. Someone found unused tensors in the server logs at 2am, tried to build the draft file, and discovered there was nothing to build. One flag connects what is already there (sudoingX found this, paired A/B, open sourced the probe before sunrise). The depth cap. The head has exactly one layer. n=4 breaks it. --spec-draft-n-max 2 n=2 is the sweet spot. n=3 is the ceiling. The model has one MTP layer, so pushing the draft depth to 4 or 5 crashes the head and it starts emitting junk tokens. People hit this on the Spark and documented the whole ladder: n=1 gives 1.75x, n=2 gives 2.37x, n=3 gives 2.85x, n=4 does not exist. Respect the cap. The memory flags. MTP brings its own luggage. --cache-type-k q8_0 --cache-type-v q8_0 --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 -np 1 Three flags, one purpose: fit it on 24GB. The KV cache is the model's running memory of your conversation, and it is the thing that eats your card at long context. q8_0 halves it with no visible quality cost. The second line does the same for the draft head's own cache, which defaults to full fat and quietly eats 2GB. And parallel slots set to 1 means requests queue instead of reserving a second pool. Single card, single lane, everything fits (AJ runs this exact trio on a 3090). The quality flag. Past 100K the model gets dumb, this is the fix. --kv-cache-dtype bfloat16 The quantized cache saves memory but degrades reasoning at long context. One person ran it all day past half the window and called the full precision fix night and day. Slight tok/s cost, real quality gain. If your sessions stay short, skip it. If you live past 100K, do not. The trap that generates "this quant is broken" reports. --jinja Qwen 3.8 ships its own chat template. Load the model without this flag and there is no reliable marker for where your turn ends and its answer begins. Two failure modes: it rambles past the stop token, or it answers clipped and loses the thread between turns. Both look like a broken quant. It is not the quant. Several packs now ship a corrected template file because the official one nests empty think blocks across turns. The Blackwell lane, if you own a 50-series or a Spark. NVFP4 instead of GGUF. The MTP flag translates to --speculative-config '{"method":"mtp","num_speculative_tokens":3}', same cap. FP8 KV cache doubles your context window (a full 1M token session costs about 32GB of cache). Two gotchas documented in the first 24 hours: stock vLLM cannot load this model's MTP architecture on a Spark, you need the community GB10 build. And FP8 KV requires a specific attention backend on the Spark, the default one silently cannot serve it. Set reasoning to medium unless you want it thinking at maximum depth on every reply. Default is xhigh and it burns your tokens. None of these came from the model card. Every one came from someone's server log, 2am session, or paired benchmark. Flip the flags, then come tell the community table what your card did. Drop in parameter flags and sources for your technical DD in reply 👇