{"record":{"id":"834c90c036e0d7a8","repo":"sgl-project/sglang","slug":"kv-cache-dtype-mxfp8-requires-an-sm100-blackwe","errorCode":null,"errorMessage":"--kv-cache-dtype mxfp8 requires an SM100+ (Blackwell) GPU for the block-scaled operands used by the FA4 MXFP8 attention path.","messagePattern":"--kv-cache-dtype mxfp8 requires an SM100\\+ \\(Blackwell\\) GPU for the block-scaled operands used by the FA4 MXFP8 attention path\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6638,"sourceCode":"            logger.warning(\n                \"Mixed chunk and radix cache are disabled when using dual-chunk flash attention backend\"\n            )\n            self._declare(\n                \"_handle_attention_backend_compatibility\",\n                enable_mixed_chunk=False,\n            )\n            self._declare(\n                \"_handle_attention_backend_compatibility\",\n                disable_radix_cache=True,\n            )\n\n    def _handle_mxfp8_kv_cache_compatibility(self):\n        \"\"\"MXFP8 KV cache uses operands available only on SM100+ (Blackwell).\"\"\"\n        cfg = resolving_view(self)\n        if cfg.kv_cache_dtype != \"mxfp8\":\n            return\n        if not is_blackwell_supported():\n            raise ValueError(\n                \"--kv-cache-dtype mxfp8 requires an SM100+ (Blackwell) GPU for the \"\n                \"block-scaled operands used by the FA4 MXFP8 attention path.\"\n            )\n\n    def _handle_kv4_compatibility(self):\n        \"\"\"Check FP4 KV cache compatibility with the attention backend\"\"\"\n        cfg = resolving_view(self)\n\n        if cfg.kv_cache_dtype not in (\"nvfp4\", \"fp4_mx_block16\"):\n            return\n\n        use_mla_backend = self.use_mla_backend()\n        prefill_backend, decode_backend = self._resolved_attention_backends()\n        attention_backend = resolved_view(self).attention_backend\n\n        if is_cuda():\n            if cfg.kv_cache_dtype == \"nvfp4\" and not (\n                is_sm100_supported() or is_sm120_supported()","sourceCodeStart":6620,"sourceCodeEnd":6656,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6620-L6656","documentation":"MXFP8 KV cache quantization relies on block-scaled FP8 MX operands (per-block scale factors) that only exist on SM100+ Blackwell GPUs and are consumed by the FA4 attention path. Server startup validates this and rejects --kv-cache-dtype mxfp8 on older hardware via is_blackwell_supported().","triggerScenarios":"Passing --kv-cache-dtype mxfp8 on any GPU that is not SM100+ (Hopper SM90, Ampere, etc.), where is_blackwell_supported() returns false.","commonSituations":"Copying Blackwell-tuned serving configs to H100 clusters; enabling mxfp8 KV cache after seeing Blackwell benchmarks; CI runners on non-Blackwell GPUs.","solutions":["Use a KV cache dtype supported on your GPU (fp8_e5m2 / fp8_e4m3 on SM90, or bf16)","Verify compute capability with torch.cuda.get_device_capability() before setting mxfp8","Run on a Blackwell (B200/GB200, SM100+) GPU if MXFP8 KV cache is required"],"exampleFix":"# before\npython -m sglang.launch_server --model M --kv-cache-dtype mxfp8   # on H100\n# after\npython -m sglang.launch_server --model M --kv-cache-dtype fp8_e5m2","handlingStrategy":"validation","validationCode":"import torch\ncap = torch.cuda.get_device_capability(0)\nsm = cap[0] * 10 + cap[1]\nif args.kv_cache_dtype == \"mxfp8\" and sm < 100:\n    args.kv_cache_dtype = \"fp8_e5m2\" if sm >= 89 else \"bf16\"","typeGuard":null,"tryCatchPattern":"try:\n    ServerArgs(**kwargs)\nexcept ValueError as e:\n    if \"mxfp8\" in str(e) and \"SM100\" in str(e):\n        kwargs[\"kv_cache_dtype\"] = \"fp8_e5m2\"\n        ServerArgs(**kwargs)\n    else:\n        raise","preventionTips":["Map kv_cache dtype tiers to GPU generations in config management (mxfp8 → Blackwell only)","Never copy benchmark configs across GPU generations unedited","Preflight: require torch.cuda.get_device_capability() >= (10, 0) before allowing mxfp8"],"tags":["sglang","kv-cache-dtype","mxfp8","blackwell","gpu-architecture"],"backgroundTag":"kv-cache-dtype-incompatible","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}