We benchmarked DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF — specifically the Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-MTP-Q6_K.gguf file — served via llama.cpp (llama-server) on a single NVIDIA RTX 5090 (32GB) with q8_0 KV cache, a 200,000-token context window (upgraded from 145K), and MTP speculative decoding (draft depth 3).
This study documents the performance envelope of the q8_0 KV / 200K build and the tradeoff the KV fidelity upgrade buys.
Peak Decode Speed
103.8 t/s
MTP @ 2K context
Decode @ Full Depth
61.9 t/s
@ 180K context (−40%)
Peak Prefill
2,254 t/s
@ 2K context (cold)
VRAM Footprint
31.9 GB
97.7% of 32GB · q8_0 KV
Model Overview
Repository: DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF
File: Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-MTP-Q6_K.gguf
Architecture:
- Base: Qwen3.6-27B hybrid — 65 blocks (17 attention + 48 Mamba/SSM), multi-stage tuned (DavidAU "Fable-Fusion 711" line — first fine-tune to exceed 700 ARC-c)
- Quantization: Q6_K weights (GGUF) — high-fidelity quant
- KV cache: q8_0 / q8_0 — doubled precision vs the previous q4_0 build
- Speculative decoding: built-in MTP draft head,
--spec-draft-n-max 3
Deployment Configuration
Hardware
- GPU: NVIDIA GeForce RTX 5090 (32GB), 500W power cap
- CPU: AMD Ryzen 9 5900X 12-Core
- RAM: 62GB DDR4
Software Stack
- Engine: llama.cpp llama-server (
/home/akshay/llama.cpp/build5/bin/llama-server) - Serve path: llama-swap → nginx (
:8001, bearer auth) - KV cache: q8_0 / q8_0 (
--cache-type-k q8_0 --cache-type-v q8_0) - Context: 200,000 tokens
- Speculative decoding:
--spec-type draft-mtp --spec-draft-n-max 3 - Flash attention:
-fa on,--jinja, reasoning enabled
Launch Configuration
llama-server \
--host 127.0.0.1 --port ${PORT} \
-ngl 99 \
-m Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-MTP-Q6_K.gguf \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--parallel 1 --jinja \
--reasoning on --reasoning-preserve \
-fa on \
--cache-type-k q8_0 --cache-type-v q8_0 \
-c 200000 \
--spec-type draft-mtp --spec-draft-n-max 3Context Window: 200K on a Single 5090
The model runs at 200,000 tokens (200K) with q8_0 KV cache — 97.7% of the 5090's 32GB VRAM (31,871 MiB used, 736 MiB free). The benchmark top-out at 180K (90%) leaves generation headroom below the window, per benchmark convention.
Why the q8_0 upgrade
- +55K context (+38%) over the previous q4_0 @ 145K build — the doubled KV precision costs ~2.6x cache memory (7,056 MiB @ 200K q8_0 vs 2,708 MiB @ 145K q4_0) but still fits on one card
- 17 attention layers only — the hybrid has just 17 KV layers (48 are Mamba/SSM with no KV cache), which is what makes 200K at q8_0 physically possible
- 100% GPU residency (
-ngl 99, 0% CPU/RAM offload) — no PCIe transfer stalls
Benchmark Results
Context Sweep
Benchmark ran via llama-benchy 0.4.0 against the llama-swap endpoint (:8001), --pp 128 --tg 100 --runs 3 --latency-mode generation. Prompts from a unique Project Gutenberg corpus — no prefix-cache reuse.
| Context | Prompt Tokens | TTFT (s) | Prefill (t/s) | Decode (t/s) | VRAM (MiB) |
|---|---|---|---|---|---|
| 1% | 2,000 | 1.05 | 2,253.9 | 103.83 | 31,871 |
| 25% | 50,000 | 23.67 | 2,126.7 | 90.46 | 31,871 |
| 50% | 100,000 | 62.62 | 1,601.6 | 75.71 | 31,871 |
| 75% | 150,000 | 115.4 | 1,302.1 | 67.9 | 31,871 |
| 90% | 180,000 | 152.2 | 1,184.3 | 61.9 | 31,871 |
Key observations:
- Decode degrades ~40% from 2K→180K: 103.8 → 61.9 t/s. Short prompts decode at 90-104 t/s; long context drops to ~62-76 t/s
- q8_0 KV costs ~10% decode at short context vs the q4_0 build (103.8 vs 115.6 t/s @ 2K) — the higher-precision cache reads are slightly heavier
- Prefill is KV-agnostic: 2,254 t/s @ 2K — actually +5% over q4_0's 2,152 t/s; prefill is weight-bandwidth-bound, not KV-bound
- TTFT scales linearly with depth: 1.05s @ 2K → 152.2s @ 180K
Full Context Sweep (raw)
| Depth | Tokens | TTFT (s) | Prefill (t/s) | Decode (t/s) |
|---|---|---|---|---|
| 1% | 2,000 | 1.05 | 2,253.9 | 103.83 |
| 25% | 50,000 | 23.67 | 2,126.7 | 90.46 |
| 50% | 100,000 | 62.62 | 1,601.6 | 75.71 |
| 75% | 150,000 | 115.40 | 1,302.1 | 67.90 |
| 90% | 180,000 | 152.20 | 1,184.3 | 61.90 |
KV Cache Tradeoff: q8_0 @ 200K vs q4_0 @ 145K
| Metric | q8_0 KV @ 200K (current) | q4_0 KV @ 145K (previous) | Δ / Note |
|---|---|---|---|
| Context window | 200,000 | 145,000 | 🏆 q8_0 (+55K, +38%) |
| Decode @ ~2K | 103.8 t/s | 115.6 t/s | q4_0 (+11%) |
| Decode @ ~50% ctx | 75.7 t/s @ 100K | 83.6 t/s @ 72.5K | q4_0 (+10%) |
| Decode @ deep ctx | 61.9 t/s @ 180K | 66.6 t/s @ 130K | q4_0 (+8%) |
| Decode degradation 2K→max | −40% | −42% | 🤝 Parity |
| Peak Prefill | 2,254 t/s @ 2K | 2,152 t/s @ 1.5K | 🏆 q8_0 (+5%) |
| KV cache size @ max | 7,056 MiB @ 200K | 2,708 MiB @ 145K | q8_0 2.6x larger |
| VRAM headroom | 736 MiB free (97.7%) | 5,334 MiB free (83.7%) | q4_0 more headroom |
Stability Notes
- Power: 5090 at 500W production cap — no power throttling observed through the full sweep at 97.7% VRAM; zero OOM events
- MTP draft at depth 3 was stable through the full sweep — no draft-acceptance collapse at long context
- KV cache is the depth bottleneck: cache reads dominate decode at 100K+ regardless of dtype — both q4_0 and q8_0 lose ~40% decode from short to full context
- VRAM margin is thin: 736 MiB free. If stability issues appear, drop to 180K (−0.7 GB KV) or fall back to q4_0 @ 200K (28.7 GB, 88% — keeps the larger window without the precision)
- Benchmark tooling:
uvx llama-benchy(v0.4.0) — token-accurate measurement, no SSE chunk-counting artifacts
Reproducibility
- Download:
hf download DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF --include "Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-MTP-Q6_K.gguf" --local-dir /home/akshay/models/davidaU--fable-fusion-711 - Serve via llama-swap (
davidaau-fable-fusion-711-q6-kv8-200-mtp) - Benchmark:
uvx llama-benchy \
--base-url http://127.0.0.1:8001/v1 \
--api-key <key> \
--model davidaau-fable-fusion-711-q6-kv8-200-mtp \
--served-model-name davidaau-fable-fusion-711-q6-kv8-200-mtp \
--tokenizer DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP \
--pp 128 --tg 100 \
--depth 2000 50000 100000 150000 180000 \
--runs 3 --latency-mode generation \
--save-result davidaau-fable-fusion-q8kv-200k-bench.json --format jsonConclusion
DavidAU Fable-Fusion-711 MTP Q6_K with q8_0 KV (llama.cpp) is a 200K-context MTP decoder on a single RTX 5090:
- 200K context stable — +38% over the q4_0 build, at 97.7% VRAM with q8_0 precision
- Decode 61.9-103.8 t/s (deep vs short context), prefill 1.2-2.3K t/s
- q8_0 KV is a context-for-speed trade: +55K window for ~10% short-context decode — degradation slope unchanged
- Hybrid architecture is the enabler: only 17 of 65 blocks carry KV cache, making 200K at full precision viable on one 32GB card
Choose the q8_0 / 200K build when document length matters more than decode latency; the model delivers the largest single-card context in the Fable-Fusion family with doubled KV fidelity.
Engineering Takeaways
- ✓q8_0 KV unlocks +55K context: 145K → 200K (+38%) on the same single RTX 5090 — the doubled-precision KV cache (7,056 MiB @ 200K vs 2,708 MiB @ 145K q4_0) fits with 736 MiB of headroom at 97.7% VRAM.
- ✓Decode pays ~10% for the fidelity: q8_0 decode runs 104 → 62 t/s (2K → 180K) vs q4_0's 116 → 67 t/s — higher-precision KV reads cost roughly a tenth of short-context decode throughput.
- ✓Degradation slope is unchanged: both KV dtypes lose ~40% decode from short to full context (q8_0 −40%, q4_0 −42%) — the depth penalty is KV-quantization-independent; it's the growing cache-read cost itself.
- ✓Prefill is KV-agnostic: q8_0 actually prefills slightly faster at short context (2,254 vs 2,152 t/s @ 2K, +5%) — prefill curves are dominated by weight bandwidth, not KV precision.
- ✓TTFT ceiling grows with the window: 1.05s @ 2K → 152.2s @ 180K. The larger context makes full-window cold starts ~1.7x slower than the 145K config — plan for ~0.85ms per context token.
- ✓97.7% VRAM is production-viable but leaves no headroom: if the rig shows stability issues at 500W with extra KV pressure, dropping to 180K (−0.7 GB KV) or q4_0 @ 200K (28.7 GB, 88%) are the escape hatches.
HuggingFace link: DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF