HF Model Benchmark: Trithemius Fable-Fusion 711 PrismAURA 5.5-bit

We benchmarked trithemius/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-MTP-PrismAura-5.5bit from HuggingFace on a single NVIDIA RTX 5090 (32GB VRAM) using vLLM with compressed-tensors backend.

This study documents the full deployment and performance characteristics of this PrismAURA 5.5-bit quantized model — from initial load to production benchmarking across context lengths.

Model Overview

Repository: trithemius/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-MTP-PrismAura-5.5bit

Architecture:

• Base: Qwen3.6-27B (Qwen3_5ForConditionalGeneration)
• Layers: 64
• Attention: GQA 24:4
• Head dimension: 256
• Hidden size: 5120
• Vision encoder: 27 layers, hidden_size=1152 (bypassed via --language-model-only)
• MTP heads: Multi-token prediction for speculative decoding
• Quantization: PrismAURA 5.5-bit mixed precision (NVFP4/FP8/BF16 compressed-tensors)
• Decoding: Heretic ARA decensored

Key features:

• Multimodal architecture (vision + text)
• Native MTP speculative decoding support
• PrismAURA 5.5-bit quantization (compressed-tensors format)
• Vision weights stored as BF16 (excluded from W4A16 quantization)

Deployment Configuration

Hardware

• GPU: NVIDIA GeForce RTX 5090 (32GB VRAM)
• CPU: AMD Ryzen 9 5900X 12-Core
• RAM: 62GB DDR4
• Storage: 477GB NVMe

Software Stack

• Engine: vLLM (compressed-tensors backend)
• Quantization: PrismAURA 5.5-bit (NVFP4/FP8/BF16 mixed)
• KV Cache: FP8 (fp8_e4m3)
• CPU Offload: 0% (--cpu-offload-gb 0)
• Vision Encoder: Disabled (--language-model-only) — saved ~2.5 GB VRAM
• Chat template: qwen-froggeric-v21.jinja with preserve_thinking enabled

Launch Configuration

vllm serve /path/to/model \
  --quantization compressed-tensors \
  --max-model-len 204800 \
  --kv-cache-memory-bytes 8100000000 \
  --kv-cache-dtype fp8_e4m3 \
  --language-model-only \
  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}' \
  --chat-template /path/to/qwen-froggeric-v21.jinja \
  --default-chat-template-kwargs '{"preserve_thinking":true}' \
  --override-generation-config '{"temperature":0.7,"top_p":0.95,"max_tokens":8192}' \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --safetensors-load-strategy prefetch \
  --compilation-config '{"cudagraph_mode":"PIECEWISE"}'

Initial Load & VRAM Analysis

First Load Attempt

First load timed out after 120s — CUDA graph compilation was in progress. Second attempt at 03:34 succeeded — model returned response with thinking/reasoning enabled.

VRAM Breakdown

• Model weights: 21.28 GiB
• KV cache budget: 8.1 GiB (215K context with MTP heads)
• Total VRAM used: 30.697 GiB / 31.843 GiB (96.4% utilization)
• CPU offload: 0% — all weights and KV cache on GPU

Context Length Optimization

MTP heads cost context window — the tradeoff for 2x decode. Both configs run fully on-GPU with 0% CPU offload:

Earlier attempts at 262K with the MTP build OOM'd on the FP8 KV cache budget — the MTP heads consume the extra ~244 MiB that would otherwise extend context by ~47K tokens.

Benchmark Results

Full Context Sweep

Benchmark ran via llama-benchy 0.4.0 (llama-bench style, OpenAI-compatible endpoints) against the llama-swap endpoint at http://localhost:8000. Prompts built from a unique Project Gutenberg corpus — no prefix-cache reuse, --latency-mode generation, --tg 100 (80 at max depth to fit context). Token counts via HuggingFace tokenizer.

Key observations:

• MTP delivers ~2x decode on single GPU: 87-103 t/s vs 42-52 t/s dense across all context levels — the speedup holds at full context (87.3 vs 42.2 t/s at max depth, no acceptance collapse)
• Prefill is unaffected by MTP: within ±4% of dense at every level (13.8K vs 14.4K t/s @ 2K; ~2.6-2.7K t/s at full context)
• Full-context stability: dense degrades ~19% from 2K→full (52.2→42.2 t/s); MTP degrades only ~13% (100.6→87.3 t/s)
• 0% CPU/RAM offload — 100% GPU at full context depth

MTP Speculative Decoding Analysis

Tested num_speculative_tokens=3 (MTP, batch 2048) vs dense execution (batch 4096):

Finding: MTP is a pure decode win — the context tradeoff is the only cost. Choose MTP for interactive/low-latency workloads; dense for maximum document length.

Correction note: earlier reports showed MTP at 31-36 t/s — that was an SSE chunk-counting artifact (vLLM packs multiple tokens per chunk under speculative decoding). Token-accurate measurement via llama-benchy shows the true ~100 t/s.

Power Limit Sweep: 500W (production) vs 600W (max)

Raised the power cap to the 600W maximum and re-ran the decode + prefill sweep to test whether the RTX 5090 is power-starved at the production 500W cap:

Verdict: 600W buys nothing measurable. Decode is flat (MTP ±5% noise: 96-108 t/s both caps; No-MTP identical 43.7-52.3). Prefill gains only +3-4% (No-MTP @2K: 14,586 vs 14,352 t/s; @196K: 3,163 vs 3,053). SM clocks pinned at 2,895 MHz under both caps — the 5090 is not power-limited at 500W for this workload. Keep the 500W cap — it avoids the fall-off-bus crash risk documented on this rig (Jul 4-6) at zero throughput cost. No crashes observed at 600W.

Reasoning & Thinking Content

vLLM returns thinking/reasoning content in the stream delta under delta['reasoning'] rather than delta['content'] or delta['reasoning_content'] when using qwen3 reasoning parser.

This is important for clients consuming streaming responses — reasoning content must be extracted from the reasoning field, not content.

Reproducibility

To reproduce this benchmark:

1. Download the model from HuggingFace:

   huggingface-cli download trithemius/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-MTP-PrismAura-5.5bit --local-dir /path/to/model

2. Serve with vLLM using the launch configuration above

3. Run benchmarks via llama-swap or directly against vLLM endpoint

4. Use llama_benchy.py script for automated context sweep testing

Conclusion

trithemius/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-MTP-PrismAura-5.5bit is a strong single-GPU choice for:

• ~2x MTP decode speedup (87-103 t/s vs 42-52 t/s dense) that holds at full context depth
• Large context windows (262K dense / 215K MTP) on a single RTX 5090
• PrismAURA 5.5-bit quantization with FP8 KV cache — 100% GPU, 0% CPU offload
• Multimodal architecture (vision + text) with vision bypass option

The MTP build trades 47K context tokens for 2x generation throughput — pick based on workload. The 500W power cap is validated as optimal: raising to 600W adds no measurable decode or prefill headroom.

HuggingFace link: trithemius/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-MTP-PrismAura-5.5bit