{"record":{"id":"4e2439217c2a401b","repo":"sgl-project/sglang","slug":"indices-must-be-int32-got-indices-dtype","errorCode":null,"errorMessage":"indices must be int32, got {indices.dtype}","messagePattern":"indices must be int32, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":370,"sourceCode":"            f\"multiple of 64, got {h_q}\"\n        )\n\n    if h_kv != 1:\n        raise ValueError(f\"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}\")\n\n    if d_qk not in (512, 576):\n        raise ValueError(\n            f\"sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got {d_qk}\"\n        )\n\n    if indices.shape[:2] != (s_q, h_kv):\n        raise ValueError(\n            \"indices must have shape \"\n            f\"({s_q}, {h_kv}, topk), got {tuple(indices.shape)}\"\n        )\n\n    if indices.dtype != torch.int32:\n        raise ValueError(f\"indices must be int32, got {indices.dtype}\")\n\n    if topk == 0 or topk % 128 != 0:\n        raise ValueError(\n            \"Q8KV8 sparse-prefill topk width must be a positive multiple of 128, \"\n            f\"got {topk}\"\n        )\n\n    if topk_length is not None:\n        if topk_length.shape != (s_q,) or topk_length.dtype != torch.int32:\n            raise ValueError(\n                f\"topk_length must be int32 with shape ({s_q},), got \"\n                f\"{tuple(topk_length.shape)}/{topk_length.dtype}\"\n            )\n        if not topk_length.is_cuda:\n            raise ValueError(\"topk_length must be a CUDA tensor\")\n        if topk_length.device != device:\n            raise ValueError(\n                \"topk_length must be on q's device \"","sourceCodeStart":352,"sourceCodeEnd":388,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L352-L388","documentation":"The kernel reads index values as 32-bit integers; indices must be torch.int32. int64 (PyTorch's default for topk/arange outputs) or int16 indices are rejected because the kernel would reinterpret them with the wrong width.","triggerScenarios":"Passing torch.topk(...).indices (int64) directly without casting to int32.","commonSituations":"Default PyTorch integer ops producing int64; porting from a backend that accepted int64; producer kernels emitting int32 but intermediate torch ops promoting to int64.","solutions":["Cast before the call: indices = indices.to(torch.int32)","Check any arithmetic done on indices (clamping, adding offsets) preserves int32 dtype","Verify the index-producing kernel/triton op writes int32 output"],"exampleFix":"// before\nindices = torch.topk(scores, k=topk, dim=-1).indices  # int64\n// after\nindices = torch.topk(scores, k=topk, dim=-1).indices.to(torch.int32)","handlingStrategy":"validation","validationCode":"if indices.dtype != torch.int32: indices = indices.to(torch.int32)","typeGuard":"def is_int32(t: torch.Tensor) -> bool:\n    return t.dtype == torch.int32","tryCatchPattern":null,"preventionTips":["Cast torch.topk outputs to int32 by convention","Make index-producing kernels emit int32"],"tags":["dtype","indices","sparse-attention"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}