{"record":{"id":"e9605912d4f5d59d","repo":"sgl-project/sglang","slug":"mxfp8-dense-gemm-requested-via-fp8-gemm-backend-e96059","errorCode":null,"errorMessage":"MXFP8 dense GEMM requested via --fp8-gemm-backend=flashinfer_cutedsl, but that kernel requires an SM100/SM103 GPU and FlashInfer.","messagePattern":"MXFP8 dense GEMM requested via --fp8-gemm-backend=flashinfer_cutedsl, but that kernel requires an SM100/SM103 GPU and FlashInfer\\.","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8_utils.py","lineNumber":592,"sourceCode":"    backend = get_fp8_gemm_runner_backend()\n\n    if backend.is_flashinfer_trtllm():\n        if not (_is_sm100_supported and is_flashinfer_available()):\n            raise RuntimeError(\n                \"MXFP8 dense GEMM requested via --fp8-gemm-backend=flashinfer_trtllm, \"\n                \"but that kernel requires SM100/SM103 GPUs and FlashInfer.\"\n            )\n        return Mxfp8DenseGemmBackend.FLASHINFER_TRTLLM\n\n    if backend.is_flashinfer_cutedsl():\n        if not (\n            is_blackwell_supported()\n            and is_flashinfer_available()\n            and _raw_flashinfer_mm_mxfp8.is_backend_supported(\n                \"cute-dsl\", get_device_sm()\n            )\n        ):\n            raise RuntimeError(\n                \"MXFP8 dense GEMM requested via --fp8-gemm-backend=flashinfer_cutedsl, \"\n                \"but that kernel requires an SM100/SM103 GPU and FlashInfer.\"\n            )\n        return Mxfp8DenseGemmBackend.FLASHINFER_CUTEDSL\n\n    if backend.is_flashinfer_cutlass():\n        if not (is_blackwell_supported() and is_flashinfer_available()):\n            raise RuntimeError(\n                \"MXFP8 dense GEMM requested via --fp8-gemm-backend=flashinfer_cutlass, \"\n                \"but that kernel requires Blackwell GPUs and FlashInfer.\"\n            )\n        return Mxfp8DenseGemmBackend.FLASHINFER_CUTLASS\n\n    if backend.is_deep_gemm():\n        if not deep_gemm_wrapper.ENABLE_JIT_DEEPGEMM:\n            raise RuntimeError(\n                \"MXFP8 dense GEMM requested via --fp8-gemm-backend=deep_gemm, but \"\n                \"DeepGEMM is not available (package missing or \"","sourceCodeStart":574,"sourceCodeEnd":610,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8_utils.py#L574-L610","documentation":"--fp8-gemm-backend=flashinfer_cutedsl for MXFP8 dense GEMM requires an SM100/SM103 GPU, FlashInfer installed, and the cute-dsl backend reporting support for the device's SM version (checked via _raw_flashinfer_mm_mxfp8.is_backend_supported). If any check fails, the explicit request raises instead of silently switching kernels.","triggerScenarios":"Setting --fp8-gemm-backend=flashinfer_cutedsl on a non-Blackwell GPU, without FlashInfer, or on an SM version the cute-dsl kernel doesn't support, while resolving the MXFP8 dense backend during layer init or dispatch.","commonSituations":"Older flashinfer wheels lacking cute-dsl MXFP8 kernels; running on Hopper with flags copied from Blackwell docs; nightly-dependent cute-dsl support gaps.","solutions":["Remove --fp8-gemm-backend (auto) so a supported MXFP8 backend is chosen","Upgrade FlashInfer to a version with cute-dsl MXFP8 support and run on SM100/SM103","Use flashinfer_cutlass or another supported backend on your GPU"],"exampleFix":"# before\n--fp8-gemm-backend flashinfer_cutedsl  # non-SM100 GPU\n# after\n--fp8-gemm-backend auto","handlingStrategy":"validation","validationCode":"import torch\nif args.fp8_gemm_backend == \"flashinfer_cutedsl\":\n    from sglang.srt.utils import is_flashinfer_available\n    assert torch.cuda.is_available() and torch.cuda.get_device_capability(0)[0] >= 10 and is_flashinfer_available(), \\\n        \"flashinfer_cutedsl mxfp8 needs SM100/SM103 + FlashInfer; use auto\"","typeGuard":"def cutedsl_mxfp8_available() -> bool:\n    import torch\n    try:\n        from sglang.srt.utils import is_flashinfer_available\n        return torch.cuda.is_available() and torch.cuda.get_device_capability(0)[0] >= 10 and is_flashinfer_available()\n    except ImportError:\n        return False","tryCatchPattern":null,"preventionTips":["Pin gemm backend flags per hardware tier in config management","Upgrade flashinfer when using cute-dsl kernels","Prefer auto resolution except for benchmarking"],"tags":["quantization","mxfp8","gemm-backend","flashinfer","hardware-compatibility"],"backgroundTag":"gemm-backend-hardware-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}