{"record":{"id":"cac2235813b9dce0","repo":"sgl-project/sglang","slug":"name-must-have-shape-tuple-shape-got-tuple","errorCode":null,"errorMessage":"{name} must have shape {tuple(shape)}, got {tuple(t.shape)}","messagePattern":"(.+?) must have shape (.+?), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":114,"sourceCode":"# ---------------------------------------------------------------------------\n\n# torch._C._cuda_getCurrentRawStream returns the cudaStream_t pointer expected\n# by the JIT wrapper. torch._C._cuda_getCurrentStream returns a packed stream\n# id and must not be used here.\n_get_current_stream_raw = torch._C._cuda_getCurrentRawStream\n\n\n# Module-level cache for kernel-write-only output tensors. The active s_q rows\n# are overwritten every call; buffers grow monotonically by device/head shape.\ndef _check_out_buffer(\n    t: torch.Tensor,\n    name: str,\n    shape: tuple,\n    dtype: torch.dtype,\n    device: torch.device,\n) -> None:\n    if tuple(t.shape) != tuple(shape):\n        raise ValueError(f\"{name} must have shape {tuple(shape)}, got {tuple(t.shape)}\")\n    if t.dtype != dtype:\n        raise ValueError(f\"{name} must have dtype {dtype}, got {t.dtype}\")\n    if t.device != device:\n        raise ValueError(f\"{name} must be on device {device}, got {t.device}\")\n    if not t.is_contiguous():\n        raise ValueError(f\"{name} must be contiguous\")\n\n\n# Internal custom-op wrappers so the JIT kernel calls participate in\n# torch.library / torch.compile tracing and kernel-API debug logging.\n# The dispatch_full variant carries the optional attn_sink / topk_length\n# tensors as required args; the public API chooses which op to call.\n@register_custom_op(\n    op_name=\"sparse_mla_q8kv8_prefill\",\n    mutates_args=[\"out\", \"max_logits\", \"lse\"],\n)\ndef _sparse_mla_q8kv8_prefill_op(\n    q: torch.Tensor,","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L96-L132","documentation":"sparse_mla_q8kv8_prefill_fwd validates caller-provided output buffers through _check_out_buffer; each buffer must exactly match the expected shape for the kernel launch. A mismatch raises ValueError naming the buffer, expected shape, and actual shape.","triggerScenarios":"Calling sparse_mla_q8kv8_prefill_fwd with an out= buffer (e.g. out/max_logits/lse) allocated with wrong dimensions — typically a stale shape from a different batch size, num heads, or sequence length.","commonSituations":"Reusing preallocated output buffers across requests with changing s_q or h_q; buffer caches keyed incompletely (missing a dim); after changing topk or head config without reallocating outputs.","solutions":["Allocate output buffers from the current inputs' shapes at call time (or use the API variant that allocates outputs)","Key any output-buffer cache on (s_q, h_q, topk, dtype) exactly","Re-check the documented shapes in the docstring (e.g. out [s_q,h_q,d_v], max_logits/lse [s_q,h_q])"],"exampleFix":"# before\nout = torch.empty(prev_s_q, h_q, d_v, ...)  # stale s_q\nsparse_mla_q8kv8_prefill_fwd(q, kv, indices, out=out, ...)\n# after\nout = torch.empty(q.shape[0], q.shape[1], d_v, dtype=torch.bfloat16, device=q.device)\nsparse_mla_q8kv8_prefill_fwd(q, kv, indices, out=out, ...)","handlingStrategy":"validation","validationCode":"expected = (s_q, h_q, d_v)\nassert tuple(out.shape) == expected, (out.shape, expected)\nsparse_mla_q8kv8_prefill_fwd(q, kv, idx, out=out, ...)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Allocate outputs per call from current input shapes","Key output-buffer caches on the full shape tuple"],"tags":["shape-validation","output-buffer","sparse-mla","fp8"],"backgroundTag":"output-buffer-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T11:17:15.048Z"}