{"record":{"id":"b093cbb0de3454c3","repo":"sgl-project/sglang","slug":"kv-must-have-shape-s-kv-h-kv-d-qk-got-tuple","errorCode":null,"errorMessage":"kv must have shape (s_kv, h_kv, d_qk), got {tuple(kv.shape)}","messagePattern":"kv must have shape \\(s_kv, h_kv, d_qk\\), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":301,"sourceCode":") -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:\n    \"\"\"Run Q8KV8 (FP8) sparse prefill attention on SM90.\n\n    The kernel writes into three output tensors. By default fresh tensors\n    are allocated and returned; callers that want to reuse buffers may pass\n    pre-allocated ``out`` / ``max_logits`` / ``lse`` tensors of the expected\n    shape/dtype/device. The three output tensors must not alias each other.\n\n    Returns:\n        out:        [s_q, h_q, d_v], bfloat16\n        max_logits: [s_q, h_q], float32\n        lse:        [s_q, h_q], float32\n    \"\"\"\n    # Validate ranks before unpacking shapes so malformed callers fail with a\n    # clear error instead of a Python unpacking/indexing exception.\n    if q.ndim != 3:\n        raise ValueError(f\"q must have shape (s_q, h_q, d_qk), got {tuple(q.shape)}\")\n    if kv.ndim != 3:\n        raise ValueError(\n            f\"kv must have shape (s_kv, h_kv, d_qk), got {tuple(kv.shape)}\"\n        )\n    if indices.ndim != 3:\n        raise ValueError(\n            \"indices must have shape (s_q, h_kv, topk), \" f\"got {tuple(indices.shape)}\"\n        )\n\n    s_q, h_q, d_qk = q.shape\n    s_kv, h_kv, kv_d_qk = kv.shape\n    topk = indices.shape[2]\n    device = q.device\n\n    # entry.cuh interprets q/kv as contiguous FP8 buffers and launches all\n    # accesses on q's CUDA device. Reject contract violations before launch.\n    if not q.is_cuda:\n        raise ValueError(\"q must be a CUDA tensor\")\n    if not kv.is_cuda:\n        raise ValueError(\"kv must be a CUDA tensor\")","sourceCodeStart":283,"sourceCodeEnd":319,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L283-L319","documentation":"Same pre-launch rank validation as for q: kv must be a 3-D (s_kv, h_kv, d_qk) tensor. Any other rank triggers ValueError including the observed shape.","triggerScenarios":"Passing a 2-D packed KV cache or 4-D batched kv to sparse_mla_q8kv8_prefill_fwd.","commonSituations":"Feeding the paged KV cache view directly without gathering to the dense [s_kv, h_kv, d] layout; porting from an API that expects (B, H, S, D).","solutions":["Gather/reshape kv to (s_kv, h_kv, d_qk) contiguous before the call","Verify the kv layout produced by your quantization/qprep step matches the kernel contract"],"exampleFix":"# before\nkv3 = kv_cache[layer]  # [num_blocks, ...] paged view\n# after\nkv3 = gather_kv_dense(kv_cache, indices_meta)  # [s_kv, h_kv, d_qk]\nout = sparse_mla_q8kv8_prefill_fwd(q, kv3, indices, ...)","handlingStrategy":"type-guard","validationCode":"assert kv.ndim == 3, f'kv must be (s_kv, h_kv, d_qk), got {kv.shape}'","typeGuard":"def is_kv3d(kv: torch.Tensor) -> bool:\n    return kv.ndim == 3","tryCatchPattern":null,"preventionTips":["Gather paged KV to dense [s_kv, h_kv, d] before sparse ops","Centralize layout conversion in one adapter function"],"tags":["rank-validation","shape-validation","sparse-mla","kv-cache"],"backgroundTag":"invalid-tensor-rank","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}