{"record":{"id":"d603e07c92b6517d","repo":"sgl-project/sglang","slug":"mixed-qkv-must-be-a-2d-tensor-got-ndim-mixed-q","errorCode":null,"errorMessage":"`mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).","messagePattern":"`mixed_qkv` must be a 2D tensor \\(got ndim=(.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent.py","lineNumber":281,"sourceCode":"    p_ht = ht + state_idx * stride_final_state_token\n    p_ht = p_ht + i_hv * V * K + o_v[:, None] * K + o_k[None, :]\n    tl.store(p_ht, b_h.to(p_ht.dtype.element_ty), mask=mask_h)\n\n\ndef fused_recurrent_gated_delta_rule_packed_decode(\n    mixed_qkv: torch.Tensor,\n    a: torch.Tensor,\n    b: torch.Tensor,\n    A_log: torch.Tensor,\n    dt_bias: torch.Tensor,\n    scale: float,\n    initial_state: torch.Tensor,\n    out: torch.Tensor,\n    ssm_state_indices: torch.Tensor,\n    use_qk_l2norm_in_kernel: bool = False,\n) -> tuple[torch.Tensor, torch.Tensor]:\n    if mixed_qkv.ndim != 2:\n        raise ValueError(\n            f\"`mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).\"\n        )\n    if mixed_qkv.stride(-1) != 1:\n        raise ValueError(\"`mixed_qkv` must be contiguous in the last dim.\")\n    if a.ndim != 2 or b.ndim != 2:\n        raise ValueError(\n            f\"`a` and `b` must be 2D tensors (got a.ndim={a.ndim}, b.ndim={b.ndim}).\"\n        )\n    if a.stride(-1) != 1 or b.stride(-1) != 1:\n        raise ValueError(\"`a`/`b` must be contiguous in the last dim.\")\n    if A_log.ndim != 1 or dt_bias.ndim != 1:\n        raise ValueError(\"`A_log`/`dt_bias` must be 1D tensors.\")\n    if A_log.stride(0) != 1 or dt_bias.stride(0) != 1:\n        raise ValueError(\"`A_log`/`dt_bias` must be contiguous.\")\n    if ssm_state_indices.ndim != 1:\n        raise ValueError(\n            f\"`ssm_state_indices` must be 1D for packed decode (got ndim={ssm_state_indices.ndim}).\"\n        )","sourceCodeStart":263,"sourceCodeEnd":299,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent.py#L263-L299","documentation":"fused_recurrent_gated_delta_rule_packed_decode requires mixed_qkv as a 2D packed tensor of shape (num_tokens, qk_dim + hv*v) — one row per decode token, Q/K/V fused along dim 1. It raises when the tensor has any other rank, e.g. the unpacked (B, T, ...) layout or separate q/k/v tensors.","triggerScenarios":"Passing a 3D/4D projection output (B, 1, qkv_dim) or (B, T, H, D) instead of a squeezed (tokens, qkv_dim) 2D tensor; passing q, k, v separately instead of the fused mixed_qkv the packed decode API expects.","commonSituations":"Migrating from the non-packed fused_recurrent API (which takes separate q/k/v with (B, H, T, D) shapes) to the packed decode variant used by sglang's hybrid attention; forgetting to .view(-1, qkv_dim) after a qkv Linear projection; benchmark/test code reusing 4D tensors.","solutions":["Reshape the fused projection to 2D: mixed_qkv.view(num_tokens, -1) or .reshape(-1, qkv_dim)","Ensure you are calling the packed decode entry point with the packed argument convention (mixed_qkv, a, b, A_log, dt_bias, initial_state, out, ssm_state_indices), not separate q/k/v"],"exampleFix":"# before\nout, state = fused_recurrent_gated_delta_rule_packed_decode(qkv, ...)  # qkv is (B, 1, D)\n# after\nout, state = fused_recurrent_gated_delta_rule_packed_decode(qkv.view(-1, qkv.shape[-1]), ...)","handlingStrategy":"validation","validationCode":"assert mixed_qkv.ndim == 2, mixed_qkv.shape\nmixed_qkv = mixed_qkv.reshape(-1, mixed_qkv.shape[-1]) if mixed_qkv.ndim != 2 else mixed_qkv","typeGuard":"def is_packed_2d(t: torch.Tensor) -> bool:\n    return t.ndim == 2","tryCatchPattern":null,"preventionTips":["Keep decode tensors flattened to (tokens, features) at the boundary between the model and kernel layers","Write one helper that packs all decode inputs and reuse it everywhere"],"tags":["fla","fused-recurrent","decode","shape-validation"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}