{"record":{"id":"721271c3acba8b55","repo":"sgl-project/sglang","slug":"kv-must-be-contiguous","errorCode":null,"errorMessage":"kv must be contiguous","messagePattern":"kv must be contiguous","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":338,"sourceCode":"    if not indices.is_cuda:\n        raise ValueError(\"indices must be a CUDA tensor\")\n\n    if kv.device != device:\n        raise ValueError(f\"kv must be on q's device {device}, got {kv.device}\")\n    if indices.device != device:\n        raise ValueError(\n            f\"indices must be on q's device {device}, got {indices.device}\"\n        )\n\n    if q.dtype != torch.float8_e4m3fn:\n        raise ValueError(f\"q must be torch.float8_e4m3fn, got {q.dtype}\")\n    if kv.dtype != torch.float8_e4m3fn:\n        raise ValueError(f\"kv must be torch.float8_e4m3fn, got {kv.dtype}\")\n\n    if not q.is_contiguous():\n        raise ValueError(\"q must be contiguous\")\n    if not kv.is_contiguous():\n        raise ValueError(\"kv must be contiguous\")\n    if not indices.is_contiguous():\n        raise ValueError(\"indices must be contiguous\")\n\n    if kv_d_qk != d_qk:\n        raise ValueError(f\"kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}\")\n\n    # The CUDA implementation uses B_H=64 and launches h_q / B_H CTAs.\n    # Reject unpadded TP-local head counts instead of launching zero CTAs and\n    # returning uninitialized outputs, which can appear to callers as a hang or\n    # a later collective failure.\n    if h_q == 0 or h_q % 64 != 0:\n        raise ValueError(\n            \"sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive \"\n            f\"multiple of 64, got {h_q}\"\n        )\n\n    if h_kv != 1:\n        raise ValueError(f\"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}\")","sourceCodeStart":320,"sourceCodeEnd":356,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L320-L356","documentation":"The kv tensor must be contiguous because the SM90 kernel reads the FP8 KV cache via raw pointer arithmetic with no stride support. A non-contiguous kv view (slicing, padding, transpose) triggers this validation error.","triggerScenarios":"Passing kv as a strided view, e.g. kv_cache[batch_start:batch_end] with non-unit stride in the last dim, or a cache laid out NHoHd that was permuted without materializing.","commonSituations":"Paged/HiCache KV layouts where a per-request view into a larger pool is non-contiguous; slicing d_qk/d_v planes out of a combined cache buffer; version changes in cache layout.","solutions":["Materialize a contiguous copy: kv = kv.contiguous() (or ensure the pool slice is contiguous)","Verify the cache layout matches what the sparse prefill backend expects (HND with unit inner stride)","Reorder/repad the cache allocation so the region passed to the kernel is dense"],"exampleFix":"// before\nout = sparse_mla_q8kv8_prefill_fwd(q, kv_view, indices)\n// after\nkv = kv_view.contiguous()\nout = sparse_mla_q8kv8_prefill_fwd(q, kv, indices)","handlingStrategy":"validation","validationCode":"if not kv.is_contiguous(): kv = kv.contiguous()","typeGuard":"def kv_ready(kv: torch.Tensor) -> bool:\n    return kv.is_contiguous() and kv.dtype == torch.float8_e4m3fn","tryCatchPattern":null,"preventionTips":["Ensure cache pool regions handed to this kernel are dense","Log kv.stride() in debug builds to catch layout drift"],"tags":["contiguity","kv-cache","cuda"],"backgroundTag":"non-contiguous-tensor","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}