@QuantDiffDotCom

Every LLM quant, diffed against the original. Measured KLD, accuracy with 95% CIs, tool calls and speed for 8–16GB GPUs and Macs. Open data.

 RTX 5070 + M2 Pro
quantdiff.com
Joined October 2026
  • Tweets20
  • Following22
  • Followers5
LM Studio's 1.1 GB Q4_K_M of LFM3-1.7B passed 164 of our 200 tool-call tests; their 1.4 GB Q6_K passed 181. On 500 math, general knowledge and science questions they tie. If you're running it for agents, the extra 0.3 GB is worth it. Full results: quantdiff.com/models/lfm3-1.…
QuantDiff chart for LFM3-1.7B, 3 lmstudio-community GGUFs. Left: accuracy on 500 GSM8K+MMLU+ARC-C questions with 95% intervals: Q4_K_M 1.1 GB 67.8%, Q6_K 1.4 GB 68.8%, Q8_0 1.8 GB 68.8%. Right: mean KLD vs the BF16 original on a log scale, lower is closer: Q4_K_M 0.0995, Q6_K 0.0106, Q8_0 0.0012.

ALT QuantDiff chart for LFM3-1.7B, 3 lmstudio-community GGUFs. Left: accuracy on 500 GSM8K+MMLU+ARC-C questions with 95% intervals: Q4_K_M 1.1 GB 67.8%, Q6_K 1.4 GB 68.8%, Q8_0 1.8 GB 68.8%. Right: mean KLD vs the BF16 original on a log scale, lower is closer: Q4_K_M 0.0995, Q6_K 0.0106, Q8_0 0.0012.

2
Replying to @bartowski1182
@bartowski1182's smallest Groot-19B-A3B file, IQ3_M at 10.7 GB, got 172 of our 200 tool-call tests right, vs 162 for his 12.5 GB Q4_K_M. On 500 questions all three scored about 80 to 81%, close enough to be luck. It fits a 12GB card. Full results: quantdiff.com/models/groot-1…
Two bar charts for Groot-19B-A3B GGUF quants from bartowski. Left: accuracy on 500 GSM8K, MMLU and ARC-Challenge questions with 95% confidence intervals: IQ3_M (10.7 GB) 80.0%, IQ4_XS (10.9 GB) 81.2%, Q4_K_M (12.5 GB) 80.6%; the intervals overlap. Right: generation speed on an Apple M2 Pro 32GB Mac: 43.9, 46.5 and 43.5 tokens per second. No KLD because the full-size reference doesn't fit in 32GB.

ALT Two bar charts for Groot-19B-A3B GGUF quants from bartowski. Left: accuracy on 500 GSM8K, MMLU and ARC-Challenge questions with 95% confidence intervals: IQ3_M (10.7 GB) 80.0%, IQ4_XS (10.9 GB) 81.2%, Q4_K_M (12.5 GB) 80.6%; the intervals overlap. Right: generation speed on an Apple M2 Pro 32GB Mac: 43.9, 46.5 and 43.5 tokens per second. No KLD because the full-size reference doesn't fit in 32GB.

3
Replying to @bartowski1182
@bartowski1182's IQ3_M of Nanbeige4.3-33B is 17.8 GB and scored 87.2% on our 500 questions, as good as his 20.8 GB Q4_K_M, so skip the bigger file. It's dense, so it's slow: about 8 tokens a second on our 32GB M2 Pro Mac. Full results: quantdiff.com/models/nanbeig…
Chart of 3 bartowski GGUF files of Nanbeige4.3-33B tested by QuantDiff. Left: accuracy on 500 questions (GSM8K, MMLU, ARC-Challenge) with 95% intervals: IQ3_M 17.8 GB 87.2% (84.0-89.8), IQ4_XS 18.6 GB 87.6% (84.4-90.2), Q4_K_M 20.8 GB 86.4% (83.1-89.1), all statistically tied. Right: generation speed on a 32GB M2 Pro Mac with llama.cpp: IQ3_M 8.0, IQ4_XS 9.5, Q4_K_M 7.9 tokens per second. No KLD: the full-size model does not fit in 32GB.

ALT Chart of 3 bartowski GGUF files of Nanbeige4.3-33B tested by QuantDiff. Left: accuracy on 500 questions (GSM8K, MMLU, ARC-Challenge) with 95% intervals: IQ3_M 17.8 GB 87.2% (84.0-89.8), IQ4_XS 18.6 GB 87.6% (84.4-90.2), Q4_K_M 20.8 GB 86.4% (83.1-89.1), all statistically tied. Right: generation speed on a 32GB M2 Pro Mac with llama.cpp: IQ3_M 8.0, IQ4_XS 9.5, Q4_K_M 7.9 tokens per second. No KLD: the full-size model does not fit in 32GB.

6
LFM2.5 24B A2B: LiquidAI's QAD-Q4_0 is the fastest of 6 quants we tested and ties the bigger ones on accuracy. QAD-Q4_0, 13.6 GB: 72.2% (95% CI 68.1–75.9), 88.9 tok/s UD-Q5_K_XL, 17.9 GB: 72.8% (68.7–76.5), 62.8 tok/s 500 Qs, M2 Pro 32GB. quantdiff.com/models/lfm2.5-…
Scatter chart of 6 LFM2.5 24B A2B GGUF quants on an Apple M2 Pro 32GB: generation speed in tok/s across, accuracy on 500 questions (GSM8K+MMLU+ARC-C) up, with 95% CI bars. LiquidAI QAD-Q4_0 (13.6 GB) sits far right at 88.9 tok/s and 72.2% (68.1–75.9). Others: Q4_K_M 14.5 GB 71.4% 78.5 tok/s; UD-Q2_K_XL 8.0 GB 70.2% 73.8 tok/s; UD-Q4_K_XL 14.9 GB 70.8% 69.1 tok/s; UD-Q3_K_XL 10.9 GB 70.0% 68.8 tok/s; UD-Q5_K_XL 17.9 GB 72.8% 62.8 tok/s. All bars overlap the best file's CI band (68.7–76.5%), so all are tied on accuracy.

ALT Scatter chart of 6 LFM2.5 24B A2B GGUF quants on an Apple M2 Pro 32GB: generation speed in tok/s across, accuracy on 500 questions (GSM8K+MMLU+ARC-C) up, with 95% CI bars. LiquidAI QAD-Q4_0 (13.6 GB) sits far right at 88.9 tok/s and 72.2% (68.1–75.9). Others: Q4_K_M 14.5 GB 71.4% 78.5 tok/s; UD-Q2_K_XL 8.0 GB 70.2% 73.8 tok/s; UD-Q4_K_XL 14.9 GB 70.8% 69.1 tok/s; UD-Q3_K_XL 10.9 GB 70.0% 68.8 tok/s; UD-Q5_K_XL 17.9 GB 72.8% 62.8 tok/s. All bars overlap the best file's CI band (68.7–76.5%), so all are tied on accuracy.

1
2
30
Sage-A1.5 (36B MoE): 4 GGUFs tested on an M2 Pro Mac. Accuracy 86.8–90.2% (500 Qs), all tied within CI. Tool calls 164–174/200. 49–51 tok/s. 16GB pick: i1-IQ3_M (14.7 GB), 88.0%. 24GB pick: IQ4_XS-exps (18.0 GB), 90.2%. No KLD (BF16 too big for 32GB). quantdiff.com/models/sage-a1…
Two bar charts for Sage-A1.5 GGUF quants. Left: accuracy on 500 GSM8K+MMLU+ARC-C questions with 95% CI: i1-Q2_K 12.3 GB 86.8%, i1-IQ3_M 14.7 GB 88.0%, IQ4_XS-exps 18.0 GB 90.2%, Q4_K_M 20.2 GB 89.8%. Right: generation speed on Apple M2 Pro 32GB (llama-bench tg128): 49.2, 50.2, 50.1, 50.9 tok/s. No KLD: BF16 reference too large for our 32GB test Mac.

ALT Two bar charts for Sage-A1.5 GGUF quants. Left: accuracy on 500 GSM8K+MMLU+ARC-C questions with 95% CI: i1-Q2_K 12.3 GB 86.8%, i1-IQ3_M 14.7 GB 88.0%, IQ4_XS-exps 18.0 GB 90.2%, Q4_K_M 20.2 GB 89.8%. Right: generation speed on Apple M2 Pro 32GB (llama-bench tg128): 49.2, 50.2, 50.1, 50.9 tok/s. No KLD: BF16 reference too large for our 32GB test Mac.

3
Flint (40B MoE): 3 GGUFs tested on an M2 Pro Mac. Accuracy 66.8–78.2% (500 Qs), @UnslothAI UD-IQ1_M (10.6 GB) to UD-Q3_K_XL (17.2 GB). Tool calls 154–166/200. UD-Q2_K_XL (12.6 GB): 75.8%, tied with Q3, 54.5 tok/s. No KLD (BF16 too big for 32GB). quantdiff.com/models/flint
QuantDiff chart: Flint, 3 unsloth GGUF quants. Accuracy UD-IQ1_M 66.8%, UD-Q2_K_XL 75.8%, UD-Q3_K_XL 78.2% (500 questions, 95% CI). Speed on Apple M2 Pro: 55.6, 54.5, 49.9 tok/s.

ALT QuantDiff chart: Flint, 3 unsloth GGUF quants. Accuracy UD-IQ1_M 66.8%, UD-Q2_K_XL 75.8%, UD-Q3_K_XL 78.2% (500 questions, 95% CI). Speed on Apple M2 Pro: 55.6, 54.5, 49.9 tok/s.

3
LFM2.5 24B A2B: 6 GGUFs tested on an M2 Pro Mac. Accuracy 70.0–72.8% (500 Qs) from @UnslothAI UD-Q2_K_XL (7.5 GB) to UD-Q5_K_XL (16.7 GB). Tool calls 162–170/200. Fastest: LiquidAI's QAD-Q4_0, 88.9 tok/s at 72.2%. No KLD (BF16 too big for 32GB). quantdiff.com/models/lfm2.5-…
Two charts for LFM2.5 24B A2B GGUF quants: accuracy 70.0 to 72.8 percent with 95% CIs, and M2 Pro generation speed 62.8 to 88.9 tok/s

ALT Two charts for LFM2.5 24B A2B GGUF quants: accuracy 70.0 to 72.8 percent with 95% CIs, and M2 Pro generation speed 62.8 to 88.9 tok/s

1
1
30
Spark-X2.6 4B: 10 GGUF quants (8 by @bartowski1182), 500-Q accuracy + M2 Pro tok/s Q8_0 4.1 GB: 78.8%, 39 tok/s Q4_K_M 2.4 GB: 78.8%, 55 IQ4_XS 2.2 GB: 75.8%, 62 Q3_K_M 1.9 GB: 72.4%, 49 IQ2_M 1.5 GB: 49.0%, 49 Maker's Q4_K_M: top tool calls, 163/200 quantdiff.com/models/spark-x…
QuantDiff chart for Spark-X2.6 4B, 10 GGUF quants. Left: accuracy on 500 items (GSM8K+MMLU+ARC-C) with 95% intervals: IQ2_M 49.0%, IQ3_XXS 67.2%, Q3_K_M 72.4%, IQ4_XS 75.8%, mradermacher i1-Q4_K_M 76.8%, bartowski Q4_K_M 78.8%, Q5_K_M 78.2%, XHToken Q4_K_M 78.6%, Q6_K 77.8%, Q8_0 78.8%. Right: mean KL divergence vs BF16 by file size, from 1.00 (IQ2_M) down to 0.0027 (Q8_0).

ALT QuantDiff chart for Spark-X2.6 4B, 10 GGUF quants. Left: accuracy on 500 items (GSM8K+MMLU+ARC-C) with 95% intervals: IQ2_M 49.0%, IQ3_XXS 67.2%, Q3_K_M 72.4%, IQ4_XS 75.8%, mradermacher i1-Q4_K_M 76.8%, bartowski Q4_K_M 78.8%, Q5_K_M 78.2%, XHToken Q4_K_M 78.6%, Q6_K 77.8%, Q8_0 78.8%. Right: mean KL divergence vs BF16 by file size, from 1.00 (IQ2_M) down to 0.0027 (Q8_0).

2
Nemotron 3.5 Bolt E4B: 8 @UnslothAI GGUF quants, 500-Q accuracy + M2 Pro tok/s UD-Q8_K_XL 7.6 GB: 81.4%, 31 tok/s UD-Q6_K_XL 6.6 GB: 81.0%, 36 UD-Q4_K_XL 4.4 GB: 80.4%, 38 UD-Q3_K_XL 3.9 GB: 80.0%, 45 UD-IQ2_M 3.3 GB: 78.0%, 42 KLD on chat-format text quantdiff.com/models/nemotro…
QuantDiff chart for Nemotron 3.5 Bolt E4B, 8 unsloth GGUF quants. Left: accuracy on 500 items (GSM8K+MMLU+ARC-C) with 95% intervals: UD-IQ2_M 78.0%, UD-IQ3_XXS 79.4%, UD-Q2_K_XL 79.2%, UD-Q3_K_XL 80.0%, UD-Q4_K_XL 80.4%, UD-Q5_K_XL 81.4%, UD-Q6_K_XL 81.0%, UD-Q8_K_XL 81.4%. Right: mean KL divergence vs BF16 by file size, measured on chat-format text, from 0.20 (UD-IQ2_M) down to 0.0009 (UD-Q8_K_XL).

ALT QuantDiff chart for Nemotron 3.5 Bolt E4B, 8 unsloth GGUF quants. Left: accuracy on 500 items (GSM8K+MMLU+ARC-C) with 95% intervals: UD-IQ2_M 78.0%, UD-IQ3_XXS 79.4%, UD-Q2_K_XL 79.2%, UD-Q3_K_XL 80.0%, UD-Q4_K_XL 80.4%, UD-Q5_K_XL 81.4%, UD-Q6_K_XL 81.0%, UD-Q8_K_XL 81.4%. Right: mean KL divergence vs BF16 by file size, measured on chat-format text, from 0.20 (UD-IQ2_M) down to 0.0009 (UD-Q8_K_XL).

5
Which quant fits your card? Pick your GPU or Mac (82 machines, every Mac Studio), get the best quant that fits, ranked by measured quality. RTX 5070 12GB @ 8k: @Alibaba_Qwen Qwen3.8 27B UD-Q2_K_XL by @UnslothAI, 87.7% (CI 83.9–90.7), 48 tok/s measured. quantdiff.com/pick/?hw=rtx-5…
QuantDiff quant picker for an NVIDIA RTX 5070 12GB at 8k context, general chat. Top pick: Qwen3.8 27B UD-Q2_K_XL by unsloth, 9.15 GB, general chat score 87.7% (84-91), 48 tok/s measured on RTX 5070. Smaller and faster, statistically tied: Qwen3.5 9B Q8_0 by unsloth, 85.7%, 66 tok/s.

ALT QuantDiff quant picker for an NVIDIA RTX 5070 12GB at 8k context, general chat. Top pick: Qwen3.8 27B UD-Q2_K_XL by unsloth, 9.15 GB, general chat score 87.7% (84-91), 48 tok/s measured on RTX 5070. Smaller and faster, statistically tied: Qwen3.5 9B Q8_0 by unsloth, 85.7%, 66 tok/s.

1
14
Load more