{"record":{"id":"9763c1308463a64a","repo":"sgl-project/sglang","slug":"out-must-have-shape-b-1-hv-v-got-out-sha","errorCode":null,"errorMessage":"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)}).","messagePattern":"`out` must have shape (.+?) \\(got out\\.shape=(.+?)\\)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent.py","lineNumber":337,"sourceCode":"        )\n\n    if initial_state.ndim != 4:\n        raise ValueError(\n            f\"`initial_state` must be a 4D tensor (got ndim={initial_state.ndim}).\"\n        )\n    if initial_state.stride(-1) != 1:\n        raise ValueError(\"`initial_state` must be contiguous in the last dim.\")\n    HV, V, K = initial_state.shape[-3:]\n    if a.shape[1] != HV or b.shape[1] != HV:\n        raise ValueError(\n            f\"`a`/`b` must have shape [B, HV] with HV={HV} (got a.shape={tuple(a.shape)}, b.shape={tuple(b.shape)}).\"\n        )\n    if A_log.numel() != HV or dt_bias.numel() != HV:\n        raise ValueError(\n            f\"`A_log` and `dt_bias` must have {HV} elements (got A_log.numel()={A_log.numel()}, dt_bias.numel()={dt_bias.numel()}).\"\n        )\n    if out.shape != (B, 1, HV, V):\n        raise ValueError(\n            f\"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)}).\"\n        )\n\n    qkv_dim = mixed_qkv.shape[1]\n    qk_dim = qkv_dim - HV * V\n    if qk_dim <= 0 or qk_dim % 2 != 0:\n        raise ValueError(\n            f\"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}.\"\n        )\n    q_dim = qk_dim // 2\n    if q_dim % K != 0:\n        raise ValueError(f\"Invalid packed Q size {q_dim}: must be divisible by K={K}.\")\n    H = q_dim // K\n    if H <= 0 or HV % H != 0:\n        raise ValueError(\n            f\"Invalid head config inferred from mixed_qkv: H={H}, HV={HV}.\"\n        )\n","sourceCodeStart":319,"sourceCodeEnd":355,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent.py#L319-L355","documentation":"The caller-preallocated output must have exactly shape (B, 1, HV, V) — one decode step per token, HV value heads, and head_dim V — where B, HV, V are all derived from the other validated inputs. The wrapper raises when out has any other shape, such as the flattened (B, HV*V) or head-first (B, HV, 1, V) layouts.","triggerScenarios":"Allocating out as (B, HV*V) to skip a later reshape; reusing a (B, T, HV, V) prefill-shaped buffer; using V and K swapped relative to the state tensor's (HV, V, K).","commonSituations":"Custom decode loops optimizing away reshapes; buffers created before a config change of num_v_heads/head_dim; porting from fla's (B, H, T, D) output convention.","solutions":["Allocate out = torch.empty((B, 1, HV, V), dtype=qkv.dtype, device=dev) with HV/V taken from initial_state.shape[-3]/shape[-2]","Flatten afterwards if a 2D result is needed: out.view(B, HV*V)"],"exampleFix":"# before\nout = torch.empty(B, HV*V, device=dev, dtype=dt)\n# after\nout = torch.empty(B, 1, HV, V, device=dev, dtype=dt)\nout, _ = fused_recurrent_gated_delta_rule_packed_decode(..., out=out, ...)\nhidden = out.view(B, HV*V)","handlingStrategy":"validation","validationCode":"B, HV, V = mixed_qkv.shape[0], initial_state.shape[-3], initial_state.shape[-2]\nexpected = (B, 1, HV, V)\nassert out.shape == expected, (out.shape, expected)","typeGuard":"def out_shape_ok(out, mixed_qkv, initial_state) -> bool:\n    return out.shape == (mixed_qkv.shape[0], 1, initial_state.shape[-3], initial_state.shape[-2])","tryCatchPattern":null,"preventionTips":["Allocate out inside a helper that reads B/HV/V from the validated inputs","Reshape to (B, HV*V) only after the kernel returns"],"tags":["fla","fused-recurrent","output-buffer","shape-validation"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}