{"record":{"id":"04688db7f17c92ee","repo":"sgl-project/sglang","slug":"q8kv8-sparse-prefill-topk-width-must-be-a-positive","errorCode":null,"errorMessage":"Q8KV8 sparse-prefill topk width must be a positive multiple of 128, got {topk}","messagePattern":"Q8KV8 sparse-prefill topk width must be a positive multiple of 128, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":373,"sourceCode":"    if h_kv != 1:\n        raise ValueError(f\"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}\")\n\n    if d_qk not in (512, 576):\n        raise ValueError(\n            f\"sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got {d_qk}\"\n        )\n\n    if indices.shape[:2] != (s_q, h_kv):\n        raise ValueError(\n            \"indices must have shape \"\n            f\"({s_q}, {h_kv}, topk), got {tuple(indices.shape)}\"\n        )\n\n    if indices.dtype != torch.int32:\n        raise ValueError(f\"indices must be int32, got {indices.dtype}\")\n\n    if topk == 0 or topk % 128 != 0:\n        raise ValueError(\n            \"Q8KV8 sparse-prefill topk width must be a positive multiple of 128, \"\n            f\"got {topk}\"\n        )\n\n    if topk_length is not None:\n        if topk_length.shape != (s_q,) or topk_length.dtype != torch.int32:\n            raise ValueError(\n                f\"topk_length must be int32 with shape ({s_q},), got \"\n                f\"{tuple(topk_length.shape)}/{topk_length.dtype}\"\n            )\n        if not topk_length.is_cuda:\n            raise ValueError(\"topk_length must be a CUDA tensor\")\n        if topk_length.device != device:\n            raise ValueError(\n                \"topk_length must be on q's device \"\n                f\"{device}, got {topk_length.device}\"\n            )\n        if not topk_length.is_contiguous():","sourceCodeStart":355,"sourceCodeEnd":391,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L355-L391","documentation":"The kernel's tile size for the sparse selection is 128, so the topk width (indices.shape[-1] or indices.shape[2]) must be a positive multiple of 128 (e.g. 128, 256, 512...). Values like 64 or 300 cannot be tiled and are rejected.","triggerScenarios":"Passing indices with last dim 64 (e.g. MLP-sparsity-style top-64), or an odd topk like 204 from a heuristic.","commonSituations":"Configuring sparse attention topk to a small value (64) or a non-multiple value expecting FlashMLA-style semantics; migrating configs from a kernel that allowed topk=64; per-request variable topk padded to a wrong width.","solutions":["Round topk up to the next multiple of 128 (e.g. 64 -> 128) and pad indices with dummy/valid indices","Only enable this q8kv8 sparse path when the configured topk is 128, 256, 384, ...","Use variable-length mode (topk_length) with padded width a multiple of 128 for intermediate effective topk values"],"exampleFix":"# before\ntopk = 64  # raises\n# after\ntopk = 128\nindices = torch.cat([indices, dummy_idx], dim=-1)  # pad width to 128","handlingStrategy":"validation","validationCode":"topk = indices.shape[-1]\nassert topk > 0 and topk % 128 == 0, f\"topk={topk} must be a positive multiple of 128\"","typeGuard":"def topk_width_ok(indices: torch.Tensor) -> bool:\n    t = indices.shape[-1]\n    return t > 0 and t % 128 == 0","tryCatchPattern":null,"preventionTips":["Pad topk up to a multiple of 128 in config validation","Use topk_length for effective lengths below the padded width"],"tags":["shape-validation","topk","sparse-attention"],"backgroundTag":"value-not-multiple-of-tile-size","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}