{"record":{"id":"db2461bf98be072b","repo":"sgl-project/sglang","slug":"hd256-forward-non-varlen-expects-q-rank-4-or-5-go","errorCode":null,"errorMessage":"hd256 forward non-varlen expects q rank 4 or 5, got rank {q_rank}","messagePattern":"hd256 forward non-varlen expects q rank 4 or 5, got rank (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/sm100_hd256_2cta_fmha_forward.py","lineNumber":283,"sourceCode":"                s_q = mQ.shape[0]\n                h_q = mQ.shape[1]\n                d = mQ.shape[2]\n            else:\n                raise RuntimeError(\n                    f\"hd256 forward varlen expects q rank 3 or 5, got rank {q_rank}\"\n                )\n        else:\n            # Non-varlen path accepts either legacy 5D tensors or standard 4D tensors.\n            if cutlass.const_expr(q_rank == 5):\n                s_q = mQ.shape[1]\n                h_q = mQ.shape[2] * mQ.shape[3]\n                d = mQ.shape[4]\n            elif cutlass.const_expr(q_rank == 4):\n                s_q = mQ.shape[1]\n                h_q = mQ.shape[2]\n                d = mQ.shape[3]\n            else:\n                raise RuntimeError(\n                    f\"hd256 forward non-varlen expects q rank 4 or 5, got rank {q_rank}\"\n                )\n\n        if cutlass.const_expr(cum_seqlen_k is not None):\n            if cutlass.const_expr(k_rank == 5):\n                s_k = mK.shape[1]\n                h_k = mK.shape[2]\n            elif cutlass.const_expr(k_rank == 3):\n                s_k = mK.shape[0]\n                h_k = mK.shape[1]\n            else:\n                raise RuntimeError(\n                    f\"hd256 forward varlen expects k rank 3 or 5, got rank {k_rank}\"\n                )\n        else:\n            if cutlass.const_expr(k_rank == 5):\n                s_k = mK.shape[1]\n                h_k = mK.shape[2]","sourceCodeStart":265,"sourceCodeEnd":301,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/sm100_hd256_2cta_fmha_forward.py#L265-L301","documentation":"The SM100 head-dim-256 non-varlen FMHA forward kernel accepts Q only as rank 4 (batch, 1, seqlen, nheads, d minus one) standard batched or rank-5 legacy layout. Any other rank raises this RuntimeError in __call__.","triggerScenarios":"Calling the hd256 forward kernel without cu_seqlens (non-varlen path) with a rank-3 (total, H, D) packed varlen tensor or a rank-2 tensor.","commonSituations":"Passing pre-packed varlen tensors but forgetting cu_seqlen_k/cu_seqlen_q so the kernel takes the non-varlen branch; reshaping mistakes that drop the batch dimension.","solutions":["Pass the full batched 4D tensor (batch, seqlen, nheads, head_dim) for the non-varlen path.","If your data is packed (total_q, H, D), supply cu_seqlen_q/cu_seqlen_k so the varlen branch is taken.","Add an assert q.dim() in (4, 5) guard before invoking the kernel."],"exampleFix":"# before (packed tensor, no cu_seqlens)\nfmha(q_packed, k_packed, v_packed)\n\n# after\nfmha(q_packed, k_packed, v_packed, cu_seqlen_q=cu_q, cu_seqlen_k=cu_k)","handlingStrategy":"validation","validationCode":"if cu_seqlen_q is None:  # non-varlen path\n    assert q.dim() in (4, 5), f\"batched Q must be rank 4 or 5, got {q.dim()}\"","typeGuard":"def is_valid_batched_q(q):\n    return q.dim() in (4, 5)","tryCatchPattern":null,"preventionTips":["Always pass cu_seqlen tensors when your data is packed varlen format.","Assert q.dim() before invoking low-level FMHA kernels."],"tags":["cuda","attention","tensor-rank","batched","head-dim-256"],"backgroundTag":"unexpected-tensor-rank","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}