{"record":{"id":"d599955ef9a4c628","repo":"sgl-project/sglang","slug":"flashinfer-sparse-mla-supports-only-glm-dsa-with-f","errorCode":null,"errorMessage":"flashinfer_sparse_mla supports only GLM DSA with FP8 KV cache on NVIDIA SM120/SM121; got model_arch={model_arch!r}, sm_major={device_sm_major}, kv_cache_dtype={kv_cache_dtype}, prefill_impl={prefill_impl!r}, decode_impl={decode_impl!r}.","messagePattern":"flashinfer_sparse_mla supports only GLM DSA with FP8 KV cache on NVIDIA SM120/SM121; got model_arch=(.+?), sm_major=(.+?), kv_cache_dtype=(.+?), prefill_impl=(.+?), decode_impl=(.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_mla_sm120.py","lineNumber":589,"sourceCode":"\ndef _validate_flashinfer_sparse_mla_backend(\n    *,\n    model_arch: str,\n    device_sm_major: int,\n    kv_cache_dtype: torch.dtype,\n    prefill_impl: str,\n    decode_impl: str,\n) -> bool:\n    selected = {prefill_impl, decode_impl}\n    uses_flashinfer_sparse_mla = \"flashinfer_sparse_mla\" in selected\n    is_glm_sm12_fp8 = (\n        model_arch in _GLM_DSA_MODEL_ARCHS\n        and device_sm_major == 12\n        and kv_cache_dtype == torch.float8_e4m3fn\n        and not _is_hip\n    )\n    if uses_flashinfer_sparse_mla and not is_glm_sm12_fp8:\n        raise ValueError(\n            \"flashinfer_sparse_mla supports only GLM DSA with FP8 KV cache \"\n            \"on NVIDIA SM120/SM121; \"\n            f\"got model_arch={model_arch!r}, sm_major={device_sm_major}, \"\n            f\"kv_cache_dtype={kv_cache_dtype}, prefill_impl={prefill_impl!r}, \"\n            f\"decode_impl={decode_impl!r}.\"\n        )\n    if is_glm_sm12_fp8:\n        unsupported = selected - {\"flashinfer_sparse_mla\"}\n        if unsupported:\n            raise ValueError(\n                \"GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports \"\n                \"only flashinfer_sparse_mla, \"\n                f\"but got {sorted(unsupported)}.\"\n            )\n    return uses_flashinfer_sparse_mla\n\n\ndef flashinfer_sparse_mla_forward(","sourceCodeStart":571,"sourceCodeEnd":607,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_mla_sm120.py#L571-L607","documentation":"The attention backend selector detected that flashinfer_sparse_mla was requested (via prefill/decode impl choice), but this implementation is only valid for GLM DSA-architecture models running on NVIDIA SM120/SM121 GPUs with an FP8 (float8_e4m3fn) KV cache. Any other combination is rejected at config validation time.","triggerScenarios":"Setting attention impl to flashinfer_sparse_mla while model_arch is not in the GLM DSA set, or device SM major != 12, or kv_cache_dtype != float8_e4m3fn, or on HIP.","commonSituations":"Defaulting a non-GLM model to sparse MLA on a 50-series (SM120) GPU; using bf16 KV cache with sparse MLA; selecting the backend on AMD.","solutions":["Switch attention backend (prefill/decode impl) away from flashinfer_sparse_mla","Or use a GLM DSA model with --kv-cache-dtype fp8_e4m3 on SM120/SM121 hardware","Verify device compute capability (needs SM major 12)"],"exampleFix":"# before\nserver_args.prefill_impl = \"flashinfer_sparse_mla\"  # bf16 cache, non-GLM model\n# after\nserver_args.prefill_impl = \"fa3\"  # or set --kv-cache-dtype fp8_e4m3 with GLM DSA","handlingStrategy":"validation","validationCode":"assert not (impl == 'flashinfer_sparse_mla' and not (arch in GLM_DSA_ARCHS and sm_major == 12 and kv_dtype == torch.float8_e4m3fn)), 'flashinfer_sparse_mla requires GLM DSA + FP8 KV + SM120'","typeGuard":"def sparse_mla_supported(model_arch: str, sm_major: int, kv_dtype: torch.dtype) -> bool:\n    return (model_arch in _GLM_DSA_MODEL_ARCHS and sm_major == 12\n            and kv_dtype == torch.float8_e4m3fn)","tryCatchPattern":"try:\n    backend = AttentionBackend(...)\nexcept ValueError as e:\n    if 'flashinfer_sparse_mla' in str(e):\n        fallback_to_default_backend()","preventionTips":["Validate backend/model/GPU/dtype matrix at server startup","Keep a compatibility table and assert against it in config"],"tags":["attention-backend","config-validation","sm120","fp8","glm"],"backgroundTag":"unsupported-backend-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}