{"record":{"id":"67e42e4cfa04f9cb","repo":"sgl-project/sglang","slug":"topk-length-must-be-int32-with-shape-s-q-got","errorCode":null,"errorMessage":"topk_length must be int32 with shape ({s_q},), got {tuple(topk_length.shape)}/{topk_length.dtype}","messagePattern":"topk_length must be int32 with shape \\((.+?),\\), got (.+?)/(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":380,"sourceCode":"\n    if indices.shape[:2] != (s_q, h_kv):\n        raise ValueError(\n            \"indices must have shape \"\n            f\"({s_q}, {h_kv}, topk), got {tuple(indices.shape)}\"\n        )\n\n    if indices.dtype != torch.int32:\n        raise ValueError(f\"indices must be int32, got {indices.dtype}\")\n\n    if topk == 0 or topk % 128 != 0:\n        raise ValueError(\n            \"Q8KV8 sparse-prefill topk width must be a positive multiple of 128, \"\n            f\"got {topk}\"\n        )\n\n    if topk_length is not None:\n        if topk_length.shape != (s_q,) or topk_length.dtype != torch.int32:\n            raise ValueError(\n                f\"topk_length must be int32 with shape ({s_q},), got \"\n                f\"{tuple(topk_length.shape)}/{topk_length.dtype}\"\n            )\n        if not topk_length.is_cuda:\n            raise ValueError(\"topk_length must be a CUDA tensor\")\n        if topk_length.device != device:\n            raise ValueError(\n                \"topk_length must be on q's device \"\n                f\"{device}, got {topk_length.device}\"\n            )\n        if not topk_length.is_contiguous():\n            raise ValueError(\"topk_length must be contiguous\")\n        if torch.any(topk_length < 0).item() or torch.any(topk_length > topk).item():\n            raise ValueError(\n                \"topk_length values must satisfy \" f\"0 <= topk_length <= topk ({topk})\"\n            )\n\n    if d_v != 512:","sourceCodeStart":362,"sourceCodeEnd":398,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L362-L398","documentation":"When the optional variable-length topk mode is used, topk_length must be an int32 tensor of shape (s_q,) holding the effective number of selected KV entries per query token. This check enforces both shape and dtype before the values are validated.","triggerScenarios":"Passing topk_length as int64, scalar, or shaped (bs, seq_len) instead of the flattened (s_q,) int32 vector.","commonSituations":"Variable topk per request from a scheduler; forgetting to flatten a batched tensor; dtype promotion from int64 counters in a producer kernel.","solutions":["Reshape to (s_q,) matching q's token count: topk_length = topk_length.reshape(-1).to(torch.int32)","Ensure the producer writes int32 per flattened prefill token","Confirm s_q (q.shape[0]) equals len(topk_length)"],"exampleFix":"// before\ntopk_length = lengths_per_batch.to(torch.int64)  # wrong dtype/shape\n// after\ntopk_length = lengths_per_batch.reshape(-1).to(torch.int32)","handlingStrategy":"validation","validationCode":"s_q = q.shape[0]\nassert topk_length.shape == (s_q,) and topk_length.dtype == torch.int32","typeGuard":"def topk_length_ok(q: torch.Tensor, tl: torch.Tensor) -> bool:\n    return tl.shape == (q.shape[0],) and tl.dtype == torch.int32","tryCatchPattern":null,"preventionTips":["Flatten and cast length tensors at production time","Keep lengths in a dedicated (s_q,) int32 buffer"],"tags":["dtype","shape-mismatch","topk"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}