{"record":{"id":"cadf788220b8c7ae","repo":"sgl-project/sglang","slug":"indices-must-be-contiguous","errorCode":null,"errorMessage":"indices must be contiguous","messagePattern":"indices must be contiguous","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":340,"sourceCode":"\n    if kv.device != device:\n        raise ValueError(f\"kv must be on q's device {device}, got {kv.device}\")\n    if indices.device != device:\n        raise ValueError(\n            f\"indices must be on q's device {device}, got {indices.device}\"\n        )\n\n    if q.dtype != torch.float8_e4m3fn:\n        raise ValueError(f\"q must be torch.float8_e4m3fn, got {q.dtype}\")\n    if kv.dtype != torch.float8_e4m3fn:\n        raise ValueError(f\"kv must be torch.float8_e4m3fn, got {kv.dtype}\")\n\n    if not q.is_contiguous():\n        raise ValueError(\"q must be contiguous\")\n    if not kv.is_contiguous():\n        raise ValueError(\"kv must be contiguous\")\n    if not indices.is_contiguous():\n        raise ValueError(\"indices must be contiguous\")\n\n    if kv_d_qk != d_qk:\n        raise ValueError(f\"kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}\")\n\n    # The CUDA implementation uses B_H=64 and launches h_q / B_H CTAs.\n    # Reject unpadded TP-local head counts instead of launching zero CTAs and\n    # returning uninitialized outputs, which can appear to callers as a hang or\n    # a later collective failure.\n    if h_q == 0 or h_q % 64 != 0:\n        raise ValueError(\n            \"sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive \"\n            f\"multiple of 64, got {h_q}\"\n        )\n\n    if h_kv != 1:\n        raise ValueError(f\"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}\")\n\n    if d_qk not in (512, 576):","sourceCodeStart":322,"sourceCodeEnd":358,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L322-L358","documentation":"The indices tensor (per-query top-k KV indices) must be contiguous int32 on the right device; this check rejects strided/sliced indices tensors because the kernel reads them with raw pointers.","triggerScenarios":"Passing indices produced by a top-k selection (torch.topk returns contiguous, but a subsequent slice/permute like indices[:, :, ::2] or indices.permute(...)) that is non-contiguous.","commonSituations":"Post-processing top-k output (sorting, deduplication, gather) with views; reusing a padded indices buffer sliced to actual topk; index tensors forwarded from a different attention path with different layout.","solutions":["Call indices = indices.contiguous() before the call","Avoid slicing/striding the indices buffer; build it at the exact (s_q, h_kv, topk) shape","Check intermediate ops (flip, narrow, expand) between topk and the kernel call"],"exampleFix":"// before\nidx = torch.sort(indices, dim=-1).values  # may be non-contiguous\nout = sparse_mla_q8kv8_prefill_fwd(q, kv, idx)\n// after\nidx = torch.sort(indices, dim=-1).values.contiguous()\nout = sparse_mla_q8kv8_prefill_fwd(q, kv, idx)","handlingStrategy":"validation","validationCode":"if not indices.is_contiguous(): indices = indices.contiguous()","typeGuard":"def indices_ready(indices: torch.Tensor) -> bool:\n    return indices.is_contiguous() and indices.dtype == torch.int32","tryCatchPattern":null,"preventionTips":["Never slice the top-k result along the last dim; rebuild at exact shape","Check contiguity after any sort/gather post-processing"],"tags":["contiguity","indices","sparse-attention"],"backgroundTag":"non-contiguous-tensor","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}