sgl-project/sglang · error · ValueError

--kv-cache-dtype mxfp8 requires an SM100+ (Blackwell) GPU fo

Error message

--kv-cache-dtype mxfp8 requires an SM100+ (Blackwell) GPU for the block-scaled operands used by the FA4 MXFP8 attention path.

What it means

MXFP8 KV cache quantization relies on block-scaled FP8 MX operands (per-block scale factors) that only exist on SM100+ Blackwell GPUs and are consumed by the FA4 attention path. Server startup validates this and rejects --kv-cache-dtype mxfp8 on older hardware via is_blackwell_supported().

Source

Thrown at python/sglang/srt/server_args.py:6638

            logger.warning(
                "Mixed chunk and radix cache are disabled when using dual-chunk flash attention backend"
            )
            self._declare(
                "_handle_attention_backend_compatibility",
                enable_mixed_chunk=False,
            )
            self._declare(
                "_handle_attention_backend_compatibility",
                disable_radix_cache=True,
            )

    def _handle_mxfp8_kv_cache_compatibility(self):
        """MXFP8 KV cache uses operands available only on SM100+ (Blackwell)."""
        cfg = resolving_view(self)
        if cfg.kv_cache_dtype != "mxfp8":
            return
        if not is_blackwell_supported():
            raise ValueError(
                "--kv-cache-dtype mxfp8 requires an SM100+ (Blackwell) GPU for the "
                "block-scaled operands used by the FA4 MXFP8 attention path."
            )

    def _handle_kv4_compatibility(self):
        """Check FP4 KV cache compatibility with the attention backend"""
        cfg = resolving_view(self)

        if cfg.kv_cache_dtype not in ("nvfp4", "fp4_mx_block16"):
            return

        use_mla_backend = self.use_mla_backend()
        prefill_backend, decode_backend = self._resolved_attention_backends()
        attention_backend = resolved_view(self).attention_backend

        if is_cuda():
            if cfg.kv_cache_dtype == "nvfp4" and not (
                is_sm100_supported() or is_sm120_supported()

View on GitHub (pinned to 0132848349)

Solutions

Use a KV cache dtype supported on your GPU (fp8_e5m2 / fp8_e4m3 on SM90, or bf16)
Verify compute capability with torch.cuda.get_device_capability() before setting mxfp8
Run on a Blackwell (B200/GB200, SM100+) GPU if MXFP8 KV cache is required

Example fix

# before
python -m sglang.launch_server --model M --kv-cache-dtype mxfp8   # on H100
# after
python -m sglang.launch_server --model M --kv-cache-dtype fp8_e5m2

Defensive patterns

Strategy: validation

Validate before calling

import torch
cap = torch.cuda.get_device_capability(0)
sm = cap[0] * 10 + cap[1]
if args.kv_cache_dtype == "mxfp8" and sm < 100:
    args.kv_cache_dtype = "fp8_e5m2" if sm >= 89 else "bf16"

Try / catch

try:
    ServerArgs(**kwargs)
except ValueError as e:
    if "mxfp8" in str(e) and "SM100" in str(e):
        kwargs["kv_cache_dtype"] = "fp8_e5m2"
        ServerArgs(**kwargs)
    else:
        raise

Prevention

Map kv_cache dtype tiers to GPU generations in config management (mxfp8 → Blackwell only)
Never copy benchmark configs across GPU generations unedited
Preflight: require torch.cuda.get_device_capability() >= (10, 0) before allowing mxfp8

When it happens

Trigger: Passing --kv-cache-dtype mxfp8 on any GPU that is not SM100+ (Hopper SM90, Ampere, etc.), where is_blackwell_supported() returns false.

Common situations: Copying Blackwell-tuned serving configs to H100 clusters; enabling mxfp8 KV cache after seeing Blackwell benchmarks; CI runners on non-Blackwell GPUs.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/834c90c036e0d7a8. Report an issue: GitHub.