{"record":{"id":"2606024ada7960c1","repo":"sgl-project/sglang","slug":"out-must-have-shape-b-1-hv-v-got-out-sha-260602","errorCode":null,"errorMessage":"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)}).","messagePattern":"`out` must have shape (.+?) \\(got out\\.shape=(.+?)\\)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/helion/kda_decode.py","lineNumber":308,"sourceCode":"            f\"dimensions (got K={K}, V={V}).\"\n        )\n    if a.shape[1] != HV * K:\n        raise ValueError(\n            f\"`a` must have shape [B, HV*K] with HV={HV}, K={K} \"\n            f\"(got a.shape={tuple(a.shape)}).\"\n        )\n    if b.shape[1] != HV:\n        raise ValueError(\n            f\"`b` must have shape [B, HV] with HV={HV} (got b.shape={tuple(b.shape)}).\"\n        )\n    if A_log.numel() != HV:\n        raise ValueError(f\"`A_log` must have {HV} elements (got {A_log.numel()}).\")\n    if dt_bias.numel() != HV * K:\n        raise ValueError(\n            f\"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()}).\"\n        )\n    if out.shape != (B, 1, HV, V):\n        raise ValueError(\n            f\"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)}).\"\n        )\n\n    qkv_dim = mixed_qkv.shape[1]\n    qk_dim = qkv_dim - HV * V\n    if qk_dim <= 0 or qk_dim % 2 != 0:\n        raise ValueError(\n            f\"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}.\"\n        )\n    q_dim = qk_dim // 2\n    if q_dim % K != 0:\n        raise ValueError(\n            f\"Invalid packed Q size {q_dim}: must be divisible by K={K}. \"\n            \"KDA packed decode requires num_q_heads == num_k_heads and \"\n            \"head_q_dim == head_k_dim.\"\n        )\n    H = q_dim // K\n    if H <= 0 or HV % H != 0:","sourceCodeStart":290,"sourceCodeEnd":326,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/helion/kda_decode.py#L290-L326","documentation":"The fused decode writes its output in place into a preallocated `out` buffer that must be exactly [B, 1, HV, V]. validate_packed_decode_inputs compares out.shape against the dims inferred from initial_state so the kernel's write addresses line up with the caller's buffer.","triggerScenarios":"Allocating out with an extra time dim (e.g. [B, T, HV, V] with T>1), transposed dims [B, 1, V, HV], or sized for a different batch than mixed_qkv.","commonSituations":"Adapting a prefill out buffer for decode; copying output allocation code from another backend with a different layout; cache reusing an out tensor allocated before batch size changed.","solutions":["Allocate out = torch.empty(B, 1, HV, V, dtype=..., device=...) right before the call","Check for swapped V/K dims — V is initial_state.shape[-2], K is shape[-1]","Never reuse a prefill-shaped buffer for the decode path"],"exampleFix":"// before\nout = torch.empty(B, seqlen, HV, V, device=dev, dtype=dt)\n// after\nout = torch.empty(B, 1, HV, V, device=dev, dtype=dt)","handlingStrategy":"validation","validationCode":"B, HV, V = mixed_qkv.size(0), initial_state.shape[-3], initial_state.shape[-2]\nout = torch.empty(B, 1, HV, V, dtype=mixed_qkv.dtype, device=mixed_qkv.device)","typeGuard":"def valid_out(out: torch.Tensor, b: int, hv: int, v: int) -> bool:\n    return out.shape == (b, 1, hv, v)","tryCatchPattern":null,"preventionTips":["Allocate `out` inside the decode step function, sized from current inputs","Never share output buffers across differing batch sizes or paths"],"tags":["kda","helion","shape-validation","output-buffer"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}