{"record":{"id":"5292e044cc11c417","repo":"sgl-project/sglang","slug":"q-must-have-shape-s-q-h-q-d-qk-got-tuple-q-s","errorCode":null,"errorMessage":"q must have shape (s_q, h_q, d_qk), got {tuple(q.shape)}","messagePattern":"q must have shape \\(s_q, h_q, d_qk\\), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":299,"sourceCode":"    max_logits: Optional[torch.Tensor] = None,  # [s_q, h_q], float32\n    lse: Optional[torch.Tensor] = None,  # [s_q, h_q], float32\n) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:\n    \"\"\"Run Q8KV8 (FP8) sparse prefill attention on SM90.\n\n    The kernel writes into three output tensors. By default fresh tensors\n    are allocated and returned; callers that want to reuse buffers may pass\n    pre-allocated ``out`` / ``max_logits`` / ``lse`` tensors of the expected\n    shape/dtype/device. The three output tensors must not alias each other.\n\n    Returns:\n        out:        [s_q, h_q, d_v], bfloat16\n        max_logits: [s_q, h_q], float32\n        lse:        [s_q, h_q], float32\n    \"\"\"\n    # Validate ranks before unpacking shapes so malformed callers fail with a\n    # clear error instead of a Python unpacking/indexing exception.\n    if q.ndim != 3:\n        raise ValueError(f\"q must have shape (s_q, h_q, d_qk), got {tuple(q.shape)}\")\n    if kv.ndim != 3:\n        raise ValueError(\n            f\"kv must have shape (s_kv, h_kv, d_qk), got {tuple(kv.shape)}\"\n        )\n    if indices.ndim != 3:\n        raise ValueError(\n            \"indices must have shape (s_q, h_kv, topk), \" f\"got {tuple(indices.shape)}\"\n        )\n\n    s_q, h_q, d_qk = q.shape\n    s_kv, h_kv, kv_d_qk = kv.shape\n    topk = indices.shape[2]\n    device = q.device\n\n    # entry.cuh interprets q/kv as contiguous FP8 buffers and launches all\n    # accesses on q's CUDA device. Reject contract violations before launch.\n    if not q.is_cuda:\n        raise ValueError(\"q must be a CUDA tensor\")","sourceCodeStart":281,"sourceCodeEnd":317,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L281-L317","documentation":"sparse_mla_q8kv8_prefill_fwd validates argument ranks before unpacking shapes. q must be a 3-D tensor (s_q, h_q, d_qk); passing any other rank raises ValueError with the actual shape so malformed calls fail clearly instead of with an unpacking exception.","triggerScenarios":"Calling sparse_mla_q8kv8_prefill_fwd with a 2-D or 4-D q (e.g. a batched [B,S,H,D] tensor or a flattened [N*D] tensor).","commonSituations":"Adapters that assume a batched attention layout; feeding the full attention hidden states instead of the reshaped per-head q; mismatches with other APIs that take 4-D qkv.","solutions":["Reshape q to (s_q, h_q, d_qk) before calling (squeeze/reshape the batch dim into s_q)","Check q.ndim == 3 in a wrapper before dispatch"],"exampleFix":"# before\nq3 = q  # shape [B, S, H, D]\nout = sparse_mla_q8kv8_prefill_fwd(q3, kv, indices, ...)\n# after\nq3 = q.reshape(-1, q.shape[2], q.shape[3])  # [B*S, H, D]\nout = sparse_mla_q8kv8_prefill_fwd(q3, kv, indices, ...)","handlingStrategy":"type-guard","validationCode":"assert q.ndim == 3, f'q must be (s_q, h_q, d_qk), got {q.shape}'","typeGuard":"def is_q3d(q: torch.Tensor) -> bool:\n    return q.ndim == 3 and q.shape[0] > 0","tryCatchPattern":null,"preventionTips":["Reshape batched [B,S,H,D] tensors to [B*S,H,D] at the adapter boundary","Validate ranks in a thin wrapper before kernel dispatch"],"tags":["rank-validation","shape-validation","sparse-mla","q8kv8"],"backgroundTag":"invalid-tensor-rank","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}