{"record":{"id":"1b8d72f6ce7cebaa","repo":"sgl-project/sglang","slug":"deepseek-v4-flashmla-sparse-q8-prefill-requires-d","errorCode":null,"errorMessage":"DeepSeek-V4 flashmla_sparse_q8 prefill requires d_v=512, got {self.head_dim_v}.","messagePattern":"DeepSeek-V4 flashmla_sparse_q8 prefill requires d_v=512, got (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/deepseek_v4_backend.py","lineNumber":582,"sourceCode":"        assert isinstance(self.token_to_kv_pool, DeepSeekV4TokenToKVPool)\n        self.c4_topk = getattr(\n            model_runner.model_config.hf_text_config, \"index_topk\", C4_TOPK\n        )\n\n        self.enable_deepseek_v4_fp4_indexer: bool = (\n            model_runner.server_args.enable_deepseek_v4_fp4_indexer\n        )\n        self.dsa_topk_backend: DSATopKBackend = DSATopKBackend.resolve(model_runner)\n        self.dsv4_prefill_backend: str = getattr(\n            model_runner.server_args, \"dsv4_prefill_backend\", \"auto\"\n        )\n        if use_dsv4_q8kv8_sparse_prefill(self.dsv4_prefill_backend):\n            if not is_sm90_supported():\n                raise ValueError(\n                    \"DeepSeek-V4 flashmla_sparse_q8 prefill requires SM90 CUDA GPUs.\"\n                )\n            if self.head_dim_v != 512:\n                raise ValueError(\n                    \"DeepSeek-V4 flashmla_sparse_q8 prefill requires d_v=512, \"\n                    f\"got {self.head_dim_v}.\"\n                )\n        self._q8kv8_qpad_buf = None\n        self._q8kv8_attn_sink_pad = None\n        self._q8kv8_identity_scale = None\n        self.topk = get_spec().speculative_eagle_topk or 0\n        assert self.topk in [0, 1], \"MTP Topk > 1 not supported for DeepSeek V4\"\n        self.mtp_enabled = self.topk > 0\n        self.speculative_num_steps = speculative_num_steps\n        self.speculative_num_draft_tokens: int = get_spec().speculative_num_draft_tokens\n        if self.speculative_num_draft_tokens is not None:\n            # Persistent target-verify metadata buffers. Allocated here (not\n            # lazily) so they are ordinary tensors: the first touch of a lazy\n            # buffer would inherit the caller's context, and a creation inside\n            # an inference_mode forward would forbid the in-place updates the\n            # graph-capture path performs outside inference mode.\n            num_reqs = self.req_to_token.shape[0]","sourceCodeStart":564,"sourceCodeEnd":600,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/deepseek_v4_backend.py#L564-L600","documentation":"The flashmla_sparse_q8 prefill kernels for DeepSeek-V4 are hardcoded for a value-head dimension of 512; the backend validates self.head_dim_v at init and raises if the model config deviates.","triggerScenarios":"Enabling the q8kv8 sparse prefill backend while the model config's head_dim_v (kv_lora_rank + qk_rope_head_dim composition) is not 512 — e.g. a DeepSeek-V4 variant or fine-tune with modified latent dimensions.","commonSituations":"Custom DeepSeek-V4 checkpoints with resized latent dims; mixing a V4 backend flag with a V3.x checkpoint whose head_dim_v differs.","solutions":["Use the default prefill backend for non-512 head_dim_v checkpoints","Verify the checkpoint's config (kv_lora_rank / rope dims) matches stock DeepSeek-V4 with d_v=512","Switch to a stock DeepSeek-V4 model when you need flashmla_sparse_q8"],"exampleFix":"# before\n--dsv4-prefill-backend flashmla_sparse_q8  # head_dim_v=256 checkpoint\n# after\n--dsv4-prefill-backend auto","handlingStrategy":"validation","validationCode":"if use_dsv4_q8kv8_sparse_prefill(prefill_backend) and head_dim_v != 512:\n    prefill_backend = \"auto\"  # don't request q8 sparse kernels","typeGuard":"def dsv4_q8_sparse_compatible(head_dim_v) -> bool:\n    return head_dim_v == 512","tryCatchPattern":null,"preventionTips":["Validate checkpoint latent dims against backend requirements in a config lint","Don't assume V4 backend flags work on resized/fine-tuned checkpoints"],"tags":["deepseek-v4","flashmla","head-dim","model-config-mismatch","sglang"],"backgroundTag":"model-config-incompatible","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}