{"record":{"id":"598c23b2449d9366","repo":"sgl-project/sglang","slug":"kv-cache-dtype-nvfp4-requires-blackwell-sm100-or","errorCode":null,"errorMessage":"--kv-cache-dtype=nvfp4 requires Blackwell SM100 or SM120. Use --kv-cache-dtype=fp4_mx_block16 for the block-size-16 FP4 recipe.","messagePattern":"--kv-cache-dtype=nvfp4 requires Blackwell SM100 or SM120\\. Use --kv-cache-dtype=fp4_mx_block16 for the block-size-16 FP4 recipe\\.","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6658,"sourceCode":"                \"block-scaled operands used by the FA4 MXFP8 attention path.\"\n            )\n\n    def _handle_kv4_compatibility(self):\n        \"\"\"Check FP4 KV cache compatibility with the attention backend\"\"\"\n        cfg = resolving_view(self)\n\n        if cfg.kv_cache_dtype not in (\"nvfp4\", \"fp4_mx_block16\"):\n            return\n\n        use_mla_backend = self.use_mla_backend()\n        prefill_backend, decode_backend = self._resolved_attention_backends()\n        attention_backend = resolved_view(self).attention_backend\n\n        if is_cuda():\n            if cfg.kv_cache_dtype == \"nvfp4\" and not (\n                is_sm100_supported() or is_sm120_supported()\n            ):\n                raise RuntimeError(\n                    \"--kv-cache-dtype=nvfp4 requires Blackwell SM100 or SM120. \"\n                    \"Use --kv-cache-dtype=fp4_mx_block16 for the block-size-16 FP4 recipe.\"\n                )\n            if (\n                prefill_backend != decode_backend and prefill_backend != \"fa4\"\n            ):  # Take care of prefill=fa4 later\n                logger.warning(\n                    f\"Attention: Using KV4 with PREFILL = {prefill_backend} \"\n                    f\"and DECODE = {decode_backend}. \"\n                    f\"Compatibility issues are unlikely, but may occur in rare edge cases.\"\n                )\n            else:\n                if prefill_backend == \"fa4\":\n                    if use_mla_backend:  # FA4 + MLA\n                        KV4_FA4_MLA_BACKEND_CHOICES = [\n                            \"cutlass_mla\",\n                            \"flashinfer\",\n                            \"trtllm_mla\",","sourceCodeStart":6640,"sourceCodeEnd":6676,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6640-L6676","documentation":"--kv-cache-dtype nvfp4 uses the NVFP4 (FP4 with per-block FP8 scales) representation whose kernels require Blackwell SM100 or SM120. On other CUDA GPUs the resolution pipeline raises a RuntimeError pointing users to the fp4_mx_block16 recipe, which works on a wider set of hardware.","triggerScenarios":"On a CUDA system where is_sm100_supported() and is_sm120_supported() are both false, passing --kv-cache-dtype nvfp4.","commonSituations":"Migrating FP4 KV-cache experimentation from a Blackwell box to Hopper; confusing the two FP4 recipes (nvfp4 vs fp4_mx_block16) when copying configs between clusters.","solutions":["Use --kv-cache-dtype fp4_mx_block16 (block-size-16 FP4 recipe) instead","Run on SM100/SM120 hardware if nvfp4 is required","Fall back to fp8 or bf16 KV cache on non-Blackwell GPUs"],"exampleFix":"# before\npython -m sglang.launch_server --model M --kv-cache-dtype nvfp4   # on H100\n# after\npython -m sglang.launch_server --model M --kv-cache-dtype fp4_mx_block16","handlingStrategy":"validation","validationCode":"import torch\ncap = torch.cuda.get_device_capability(0)\nsm = cap[0] * 10 + cap[1]\nif args.kv_cache_dtype == \"nvfp4\" and sm not in (100, 120):\n    args.kv_cache_dtype = \"fp4_mx_block16\"  # portable block-16 FP4 recipe","typeGuard":null,"tryCatchPattern":"try:\n    ServerArgs(**kwargs)\nexcept RuntimeError as e:\n    if \"nvfp4\" in str(e):\n        kwargs[\"kv_cache_dtype\"] = \"fp4_mx_block16\"\n        ServerArgs(**kwargs)\n    else:\n        raise","preventionTips":["Know the two FP4 recipes: nvfp4 (SM100/120 only) vs fp4_mx_block16 (portable)","Select kv-cache dtype from a hardware capability lookup, not a hardcoded string","Document per-cluster supported KV dtypes to avoid config drift"],"tags":["sglang","nvfp4","kv-cache-dtype","blackwell","gpu-architecture"],"backgroundTag":"kv-cache-dtype-incompatible","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}