{"record":{"id":"cd10285b9d5dc8a4","repo":"sgl-project/sglang","slug":"deepseek-v4-flashmla-sparse-q8-prefill-requires-sm","errorCode":null,"errorMessage":"DeepSeek-V4 flashmla_sparse_q8 prefill requires SM90 CUDA GPUs.","messagePattern":"DeepSeek-V4 flashmla_sparse_q8 prefill requires SM90 CUDA GPUs\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/deepseek_v4_backend.py","lineNumber":578,"sourceCode":"        self.hisparse_coordinator = model_runner.hisparse_coordinator\n        self.req_to_token = model_runner.req_to_token_pool.req_to_token\n        self.MAX_SEQ_LEN_FOR_CAPTURE = self.req_to_token.shape[1]\n\n        assert isinstance(self.token_to_kv_pool, DeepSeekV4TokenToKVPool)\n        self.c4_topk = getattr(\n            model_runner.model_config.hf_text_config, \"index_topk\", C4_TOPK\n        )\n\n        self.enable_deepseek_v4_fp4_indexer: bool = (\n            model_runner.server_args.enable_deepseek_v4_fp4_indexer\n        )\n        self.dsa_topk_backend: DSATopKBackend = DSATopKBackend.resolve(model_runner)\n        self.dsv4_prefill_backend: str = getattr(\n            model_runner.server_args, \"dsv4_prefill_backend\", \"auto\"\n        )\n        if use_dsv4_q8kv8_sparse_prefill(self.dsv4_prefill_backend):\n            if not is_sm90_supported():\n                raise ValueError(\n                    \"DeepSeek-V4 flashmla_sparse_q8 prefill requires SM90 CUDA GPUs.\"\n                )\n            if self.head_dim_v != 512:\n                raise ValueError(\n                    \"DeepSeek-V4 flashmla_sparse_q8 prefill requires d_v=512, \"\n                    f\"got {self.head_dim_v}.\"\n                )\n        self._q8kv8_qpad_buf = None\n        self._q8kv8_attn_sink_pad = None\n        self._q8kv8_identity_scale = None\n        self.topk = get_spec().speculative_eagle_topk or 0\n        assert self.topk in [0, 1], \"MTP Topk > 1 not supported for DeepSeek V4\"\n        self.mtp_enabled = self.topk > 0\n        self.speculative_num_steps = speculative_num_steps\n        self.speculative_num_draft_tokens: int = get_spec().speculative_num_draft_tokens\n        if self.speculative_num_draft_tokens is not None:\n            # Persistent target-verify metadata buffers. Allocated here (not\n            # lazily) so they are ordinary tensors: the first touch of a lazy","sourceCodeStart":560,"sourceCodeEnd":596,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/deepseek_v4_backend.py#L560-L596","documentation":"The DeepSeek-V4 flashmla_sparse_q8 prefill path (quantized KV, sparse MLA) is built on SM90 (Hopper) kernels. The backend __init__ checks is_sm90_supported() and rejects the configuration on non-SM90 GPUs before any capture or inference.","triggerScenarios":"Setting server_args.dsv4_prefill_backend to the flashmla_sparse_q8 (q8kv8 sparse) mode — or resolving to it via 'auto' on a qualifying config — on a GPU that is not compute capability 9.0 (e.g. A100 SM80, Ada SM89, or non-CUDA devices).","commonSituations":"Borrowing a DeepSeek-V4 launch config tuned on H100 and running it on A100/L40S; enabling the q8 sparse prefill backend flag for memory savings on older hardware.","solutions":["Remove or change the dsv4_prefill_backend override to a backend supported on your GPU (e.g. default/auto non-q8 path)","Run on an SM90 (H100/H200) GPU to use flashmla_sparse_q8"],"exampleFix":"# before\n--dsv4-prefill-backend flashmla_sparse_q8  # on A100\n# after\n--dsv4-prefill-backend auto","handlingStrategy":"validation","validationCode":"from sglang.srt.utils import is_sm90_supported\nif use_dsv4_q8kv8_sparse_prefill(getattr(server_args, \"dsv4_prefill_backend\", \"auto\")) and not is_sm90_supported():\n    server_args.dsv4_prefill_backend = \"auto\"  # downgrade before launch","typeGuard":"def supports_q8_sparse_prefill() -> bool:\n    import torch\n    return torch.cuda.is_available() and torch.cuda.get_device_capability()[0] == 9","tryCatchPattern":null,"preventionTips":["Auto-downgrade hardware-specific backend flags based on device capability at startup","Keep per-GPU-generation launch configs (H100 vs A100) separate"],"tags":["deepseek-v4","flashmla","q8-kv","sm90","gpu-compatibility","sglang"],"backgroundTag":"gpu-arch-not-supported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}