{"record":{"id":"f9df9cd748354eeb","repo":"sgl-project/sglang","slug":"sparse-mla-q8kv8-prefill-fwd-only-supports-d-v-512","errorCode":null,"errorMessage":"sparse_mla_q8kv8_prefill_fwd only supports d_v=512, got {d_v}","messagePattern":"sparse_mla_q8kv8_prefill_fwd only supports d_v=512, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":399,"sourceCode":"                f\"topk_length must be int32 with shape ({s_q},), got \"\n                f\"{tuple(topk_length.shape)}/{topk_length.dtype}\"\n            )\n        if not topk_length.is_cuda:\n            raise ValueError(\"topk_length must be a CUDA tensor\")\n        if topk_length.device != device:\n            raise ValueError(\n                \"topk_length must be on q's device \"\n                f\"{device}, got {topk_length.device}\"\n            )\n        if not topk_length.is_contiguous():\n            raise ValueError(\"topk_length must be contiguous\")\n        if torch.any(topk_length < 0).item() or torch.any(topk_length > topk).item():\n            raise ValueError(\n                \"topk_length values must satisfy \" f\"0 <= topk_length <= topk ({topk})\"\n            )\n\n    if d_v != 512:\n        raise ValueError(\n            f\"sparse_mla_q8kv8_prefill_fwd only supports d_v=512, got {d_v}\"\n        )\n\n    if attn_sink is not None and topk_length is None:\n        raise ValueError(\"attn_sink requires topk_length to be provided as well\")\n\n    if attn_sink is not None:\n        if attn_sink.shape != (h_q,) or attn_sink.dtype != torch.float32:\n            raise ValueError(\n                f\"attn_sink must be float32 with shape ({h_q},), got \"\n                f\"{tuple(attn_sink.shape)}/{attn_sink.dtype}\"\n            )\n        if not attn_sink.is_cuda:\n            raise ValueError(\"attn_sink must be a CUDA tensor\")\n        if attn_sink.device != device:\n            raise ValueError(\n                f\"attn_sink must be on q's device {device}, got {attn_sink.device}\"\n            )","sourceCodeStart":381,"sourceCodeEnd":417,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L381-L417","documentation":"This sparse MLA kernel only supports value head dim d_v=512 (the DeepSeek latent value width). Unlike d_qk (512 or 576), the value projection width is fixed at 512, and other values are rejected.","triggerScenarios":"Passing a q/o layout where the value dim derived from the tensors is 576 (e.g. including rope carriers in d_v) or 256/1024.","commonSituations":"Splitting heads incorrectly so rope dims leak into d_v; non-DeepSeek models with different v_head_dim; config using qk dims for the value path.","solutions":["Ensure the value (output) head dim passed to the kernel is exactly 512","For DeepSeek-style models, split q into qk (512/576) and keep v/latent at 512","Use a different backend if your model genuinely has v_head_dim != 512"],"exampleFix":"# before\nq layout implies d_v=576 (rope carriers included)\n# after\nq_nope_rope: d_qk=576; value path uses d_v=512 latent","handlingStrategy":"validation","validationCode":"assert d_v == 512, f\"d_v must be 512, got {d_v}\"","typeGuard":"def supported_d_v(d_v: int) -> bool:\n    return d_v == 512","tryCatchPattern":null,"preventionTips":["Split qk and value head dims explicitly in the projection","Validate dims once at backend init"],"tags":["shape-validation","mla","unsupported-dim"],"backgroundTag":"unsupported-head-dimension","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}