{"record":{"id":"01d91ef5e3ec0d00","repo":"sgl-project/sglang","slug":"sparse-mla-q8kv8-prefill-fwd-requires-h-kv-1-got","errorCode":null,"errorMessage":"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}","messagePattern":"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":356,"sourceCode":"        raise ValueError(\"kv must be contiguous\")\n    if not indices.is_contiguous():\n        raise ValueError(\"indices must be contiguous\")\n\n    if kv_d_qk != d_qk:\n        raise ValueError(f\"kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}\")\n\n    # The CUDA implementation uses B_H=64 and launches h_q / B_H CTAs.\n    # Reject unpadded TP-local head counts instead of launching zero CTAs and\n    # returning uninitialized outputs, which can appear to callers as a hang or\n    # a later collective failure.\n    if h_q == 0 or h_q % 64 != 0:\n        raise ValueError(\n            \"sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive \"\n            f\"multiple of 64, got {h_q}\"\n        )\n\n    if h_kv != 1:\n        raise ValueError(f\"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}\")\n\n    if d_qk not in (512, 576):\n        raise ValueError(\n            f\"sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got {d_qk}\"\n        )\n\n    if indices.shape[:2] != (s_q, h_kv):\n        raise ValueError(\n            \"indices must have shape \"\n            f\"({s_q}, {h_kv}, topk), got {tuple(indices.shape)}\"\n        )\n\n    if indices.dtype != torch.int32:\n        raise ValueError(f\"indices must be int32, got {indices.dtype}\")\n\n    if topk == 0 or topk % 128 != 0:\n        raise ValueError(\n            \"Q8KV8 sparse-prefill topk width must be a positive multiple of 128, \"","sourceCodeStart":338,"sourceCodeEnd":374,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L338-L374","documentation":"This sparse MLA kernel only supports multi-head attention with a single shared KV head (MLA-style, h_kv=1) where all query heads attend to one latent KV stream. A kv tensor with h_kv != 1 is rejected.","triggerScenarios":"Passing a kv cache with a leading head dim greater than 1, e.g. shape (s_kv, h_kv=8, d_qk) from a grouped-query attention layout.","commonSituations":"Reusing this MLA-specific kernel with a GQA/MQA model; cache shape assumptions from a different backend where the head dim is folded differently.","solutions":["Ensure the kv cache is squeezed to the single shared latent head (h_kv=1) as MLA produces","Use the appropriate GQA attention backend for models with multiple KV heads","Check that kv.shape is (s_kv, 1, d_qk...) before calling"],"exampleFix":"// before\nkv = kv_cache  # shape (s_kv, 8, d)\nout = sparse_mla_q8kv8_prefill_fwd(q, kv, indices)\n// after\nassert kv_cache.shape[1] == 1\nout = sparse_mla_q8kv8_prefill_fwd(q, kv_cache, indices)","handlingStrategy":"validation","validationCode":"assert kv.shape[1] == 1, f\"h_kv must be 1 for sparse MLA, got {kv.shape[1]}\"","typeGuard":"def is_mla_kv(kv: torch.Tensor) -> bool:\n    return kv.shape[1] == 1","tryCatchPattern":null,"preventionTips":["Only route MLA models to this backend","Check cache layout has a singleton head dim"],"tags":["shape-validation","mla","sparse-attention"],"backgroundTag":"unsupported-kv-head-count","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}