{"record":{"id":"2a7079e3535eeff3","repo":"sgl-project/sglang","slug":"glm-dsa-with-fp8-kv-cache-on-nvidia-sm120-sm121-su","errorCode":null,"errorMessage":"GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports only flashinfer_sparse_mla, but got {sorted(unsupported)}.","messagePattern":"GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports only flashinfer_sparse_mla, but got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_mla_sm120.py","lineNumber":599,"sourceCode":"    uses_flashinfer_sparse_mla = \"flashinfer_sparse_mla\" in selected\n    is_glm_sm12_fp8 = (\n        model_arch in _GLM_DSA_MODEL_ARCHS\n        and device_sm_major == 12\n        and kv_cache_dtype == torch.float8_e4m3fn\n        and not _is_hip\n    )\n    if uses_flashinfer_sparse_mla and not is_glm_sm12_fp8:\n        raise ValueError(\n            \"flashinfer_sparse_mla supports only GLM DSA with FP8 KV cache \"\n            \"on NVIDIA SM120/SM121; \"\n            f\"got model_arch={model_arch!r}, sm_major={device_sm_major}, \"\n            f\"kv_cache_dtype={kv_cache_dtype}, prefill_impl={prefill_impl!r}, \"\n            f\"decode_impl={decode_impl!r}.\"\n        )\n    if is_glm_sm12_fp8:\n        unsupported = selected - {\"flashinfer_sparse_mla\"}\n        if unsupported:\n            raise ValueError(\n                \"GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports \"\n                \"only flashinfer_sparse_mla, \"\n                f\"but got {sorted(unsupported)}.\"\n            )\n    return uses_flashinfer_sparse_mla\n\n\ndef flashinfer_sparse_mla_forward(\n    q: torch.Tensor,\n    kv_cache: torch.Tensor,\n    indices: torch.Tensor,\n    seq_lens: torch.Tensor,\n    workspace_buffer: torch.Tensor,\n    *,\n    page_size: int,\n    kv_cache_dim: int,\n    qk_nope_head_dim: int,\n    kv_lora_rank: int,","sourceCodeStart":581,"sourceCodeEnd":617,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_mla_sm120.py#L581-L617","documentation":"The inverse constraint of error 327: when running GLM DSA with FP8 KV cache on SM120/SM121, the only supported attention implementation is flashinfer_sparse_mla. The validator rejects any other selected impl for this hardware/model/dtype combination.","triggerScenarios":"Running GLM DSA with kv_cache_dtype=fp8_e4m3 on SM120 while selected impls include anything besides flashinfer_sparse_mla.","commonSituations":"Explicitly overriding prefill/decode impl to fa3/flashmla/triton for GLM on a 50-series GPU; stale config from another model carried over.","solutions":["Set the attention impl to flashinfer_sparse_mla (remove overrides)","Or change kv_cache_dtype back to bf16/fp16 if another impl is required","Ensure the sparse-mla backend is installed/enabled in the build"],"exampleFix":"# before\nserver_args.decode_impl = \"flashmla\"  # GLM DSA + fp8 on SM120\n# after\nserver_args.decode_impl = \"flashinfer_sparse_mla\"","handlingStrategy":"validation","validationCode":"if is_glm_sm12_fp8:\n    assert selected <= {'flashinfer_sparse_mla'}, f'unsupported impls for GLM+FP8+SM120: {selected}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't override attention impl manually for GLM DSA on SM120 with FP8","Let the backend auto-select defaults unless you know the constraint matrix"],"tags":["attention-backend","config-validation","sm120","fp8","glm"],"backgroundTag":"unsupported-backend-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}