{"record":{"id":"aecec49ccbe9ce52","repo":"sgl-project/sglang","slug":"indices-must-have-shape-s-q-h-kv-topk-got","errorCode":null,"errorMessage":"indices must have shape ({s_q}, {h_kv}, topk), got {tuple(indices.shape)}","messagePattern":"indices must have shape \\((.+?), (.+?), topk\\), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":364,"sourceCode":"    # Reject unpadded TP-local head counts instead of launching zero CTAs and\n    # returning uninitialized outputs, which can appear to callers as a hang or\n    # a later collective failure.\n    if h_q == 0 or h_q % 64 != 0:\n        raise ValueError(\n            \"sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive \"\n            f\"multiple of 64, got {h_q}\"\n        )\n\n    if h_kv != 1:\n        raise ValueError(f\"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}\")\n\n    if d_qk not in (512, 576):\n        raise ValueError(\n            f\"sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got {d_qk}\"\n        )\n\n    if indices.shape[:2] != (s_q, h_kv):\n        raise ValueError(\n            \"indices must have shape \"\n            f\"({s_q}, {h_kv}, topk), got {tuple(indices.shape)}\"\n        )\n\n    if indices.dtype != torch.int32:\n        raise ValueError(f\"indices must be int32, got {indices.dtype}\")\n\n    if topk == 0 or topk % 128 != 0:\n        raise ValueError(\n            \"Q8KV8 sparse-prefill topk width must be a positive multiple of 128, \"\n            f\"got {topk}\"\n        )\n\n    if topk_length is not None:\n        if topk_length.shape != (s_q,) or topk_length.dtype != torch.int32:\n            raise ValueError(\n                f\"topk_length must be int32 with shape ({s_q},), got \"\n                f\"{tuple(topk_length.shape)}/{topk_length.dtype}\"","sourceCodeStart":346,"sourceCodeEnd":382,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L346-L382","documentation":"indices must have leading shape (s_q, h_kv=1), i.e. one index row per query token for the single latent KV head, with the last dim being topk. This check compares indices.shape[:2] against q's s_q and the required h_kv.","triggerScenarios":"Passing indices shaped for a decode batch (e.g. (bs, 1, topk) when s_q is the full prefill token count), or with head dim > 1.","commonSituations":"Reusing decode-path topk indices for prefill; producer that emits indices per-batch instead of per-token; mismatch between s_q used in q (flattened tokens) and indices (batched).","solutions":["Regenerate indices at shape (s_q, 1, topk) matching the flattened prefill token count of q","Verify the top-k index producer runs over the same flattened token dimension as q","Print q.shape[0] and indices.shape to confirm they agree"],"exampleFix":"# before\nindices.shape == (bs, 1, topk); s_q = bs * seq_len\n# after\nindices = indices.reshape(s_q, 1, topk).contiguous()","handlingStrategy":"validation","validationCode":"s_q = q.shape[0]\nassert indices.shape[:2] == (s_q, 1), f\"indices {tuple(indices.shape)} vs s_q={s_q}\"","typeGuard":"def indices_shape_ok(q: torch.Tensor, indices: torch.Tensor) -> bool:\n    return tuple(indices.shape[:2]) == (q.shape[0], 1)","tryCatchPattern":null,"preventionTips":["Produce indices on the same flattened token axis as q","Flatten batch dims consistently between q and indices"],"tags":["shape-mismatch","indices","sparse-attention"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}