{"record":{"id":"6357225f152c6626","repo":"sgl-project/sglang","slug":"sparse-mla-q8kv8-prefill-fwd-supports-d-qk-512-576","errorCode":null,"errorMessage":"sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got {d_qk}","messagePattern":"sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":359,"sourceCode":"\n    if kv_d_qk != d_qk:\n        raise ValueError(f\"kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}\")\n\n    # The CUDA implementation uses B_H=64 and launches h_q / B_H CTAs.\n    # Reject unpadded TP-local head counts instead of launching zero CTAs and\n    # returning uninitialized outputs, which can appear to callers as a hang or\n    # a later collective failure.\n    if h_q == 0 or h_q % 64 != 0:\n        raise ValueError(\n            \"sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive \"\n            f\"multiple of 64, got {h_q}\"\n        )\n\n    if h_kv != 1:\n        raise ValueError(f\"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}\")\n\n    if d_qk not in (512, 576):\n        raise ValueError(\n            f\"sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got {d_qk}\"\n        )\n\n    if indices.shape[:2] != (s_q, h_kv):\n        raise ValueError(\n            \"indices must have shape \"\n            f\"({s_q}, {h_kv}, topk), got {tuple(indices.shape)}\"\n        )\n\n    if indices.dtype != torch.int32:\n        raise ValueError(f\"indices must be int32, got {indices.dtype}\")\n\n    if topk == 0 or topk % 128 != 0:\n        raise ValueError(\n            \"Q8KV8 sparse-prefill topk width must be a positive multiple of 128, \"\n            f\"got {topk}\"\n        )\n","sourceCodeStart":341,"sourceCodeEnd":377,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L341-L377","documentation":"The FP8 sparse prefill kernel is templated only for QK head dimensions 512 and 576 (the DeepSeek-MLA latent dims: 512 + optional 64 rope carriers). Any other d_qk is rejected because no compiled kernel specialization exists.","triggerScenarios":"q with per-head QK dim 128, 256, or 1024 passed to sparse_mla_q8kv8_prefill_fwd.","commonSituations":"Trying to reuse the SM90 sparse MLA kernel with a non-DeepSeek model; config picking up v_head_dim or kv_lora_rank incorrectly as the head dim.","solutions":["Confirm the model's qk_nope_head_dim + qk_rope_head_dim equals 512 or 576","If not, this kernel is not applicable; fall back to FlashMLA/FlashInfer/triton prefill for that model","Check that d_qk is derived from q.shape (last dim), not from a stale config value"],"exampleFix":"# before\nmodel has qk dims 128+64=192 -> kernel raises\n# after: route to a generic backend\nattn_backend = 'fa3'  # instead of q8kv8 sparse prefill","handlingStrategy":"validation","validationCode":"d_qk = q.shape[-1]\nassert d_qk in (512, 576), f\"unsupported d_qk {d_qk}\"","typeGuard":"def supported_d_qk(q: torch.Tensor) -> bool:\n    return q.shape[-1] in (512, 576)","tryCatchPattern":null,"preventionTips":["Gate backend selection on model config dims at init, not per forward","Keep a dim->backend mapping table"],"tags":["shape-validation","mla","unsupported-dim"],"backgroundTag":"unsupported-head-dimension","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}