{"record":{"id":"6639f6aa31b4dd8d","repo":"sgl-project/sglang","slug":"q-must-be-contiguous","errorCode":null,"errorMessage":"q must be contiguous","messagePattern":"q must be contiguous","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":336,"sourceCode":"    if not kv.is_cuda:\n        raise ValueError(\"kv must be a CUDA tensor\")\n    if not indices.is_cuda:\n        raise ValueError(\"indices must be a CUDA tensor\")\n\n    if kv.device != device:\n        raise ValueError(f\"kv must be on q's device {device}, got {kv.device}\")\n    if indices.device != device:\n        raise ValueError(\n            f\"indices must be on q's device {device}, got {indices.device}\"\n        )\n\n    if q.dtype != torch.float8_e4m3fn:\n        raise ValueError(f\"q must be torch.float8_e4m3fn, got {q.dtype}\")\n    if kv.dtype != torch.float8_e4m3fn:\n        raise ValueError(f\"kv must be torch.float8_e4m3fn, got {kv.dtype}\")\n\n    if not q.is_contiguous():\n        raise ValueError(\"q must be contiguous\")\n    if not kv.is_contiguous():\n        raise ValueError(\"kv must be contiguous\")\n    if not indices.is_contiguous():\n        raise ValueError(\"indices must be contiguous\")\n\n    if kv_d_qk != d_qk:\n        raise ValueError(f\"kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}\")\n\n    # The CUDA implementation uses B_H=64 and launches h_q / B_H CTAs.\n    # Reject unpadded TP-local head counts instead of launching zero CTAs and\n    # returning uninitialized outputs, which can appear to callers as a hang or\n    # a later collective failure.\n    if h_q == 0 or h_q % 64 != 0:\n        raise ValueError(\n            \"sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive \"\n            f\"multiple of 64, got {h_q}\"\n        )\n","sourceCodeStart":318,"sourceCodeEnd":354,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L318-L354","documentation":"sparse_mla_q8kv8_prefill_fwd requires q to be a contiguous (dense, row-major) tensor because the CUDA kernel indexes q with raw pointer arithmetic assuming a contiguous layout. Non-contiguous q (e.g. a sliced or transposed view) is rejected before launch.","triggerScenarios":"Passing q as a slice like q[:, :, :512] or a permuted/transposed view that yields is_contiguous()==False.","commonSituations":"Splitting head dims (d_qk 512 vs 576 + d_v 512) by slicing; using q.transpose(...) or narrow views from a fused projection output; per-token FP8 quantization helpers that return strided views.","solutions":["Call q = q.contiguous() before invoking the kernel","If slicing caused it, materialize the slice with .clone() or rework the projection to output contiguous tensors","Audit any view/permute applied to q between the linear layer and the attention call"],"exampleFix":"// before\nout = sparse_mla_q8kv8_prefill_fwd(q[:, :, :576].contiguous-like_view, ...)\n// after\nq = q[:, :, :576].contiguous()\nout = sparse_mla_q8kv8_prefill_fwd(q, ...)","handlingStrategy":"validation","validationCode":"if not q.is_contiguous(): q = q.contiguous()","typeGuard":"def contiguous_or_fix(t: torch.Tensor) -> torch.Tensor:\n    return t if t.is_contiguous() else t.contiguous()","tryCatchPattern":null,"preventionTips":["Avoid slicing/transposing q right before attention; materialize views once","Wrap kernel call sites with a small contract-check helper"],"tags":["contiguity","cuda","sparse-attention"],"backgroundTag":"non-contiguous-tensor","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}