{"record":{"id":"5908a3f4d613ff6e","repo":"sgl-project/sglang","slug":"the-batch-size-is-expected-to-be-1-rather-than-q","errorCode":null,"errorMessage":"The batch size is expected to be 1 rather than {q.shape[0]} when using `cu_seqlens`.Please flatten variable-length inputs before processing.","messagePattern":"The batch size is expected to be 1 rather than (.+?) when using `cu_seqlens`\\.Please flatten variable-length inputs before processing\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/chunk.py","lineNumber":234,"sourceCode":"\n    if head_first:\n        raise DeprecationWarning(\n            \"head_first is deprecated and will be removed in a future version. \"\n            \"Please use head_first=False for now instead.\"\n        )\n        q, k, v, beta, g = map(\n            lambda x: rearrange(x, \"b h t ... -> b t h ...\"), (q, k, v, beta, g)\n        )\n    # if not head_first and q.shape[1] < q.shape[2]:\n    #     warnings.warn(\n    #         f\"Input tensor shape suggests potential format mismatch: seq_len ({q.shape[1]}) < num_heads ({q.shape[2]}). \"\n    #         \"This may indicate the inputs were passed in head-first format [B, H, T, ...] \"\n    #         \"when head_first=False was specified. \"\n    #         \"Please verify your input tensor format matches the expected shape [B, T, H, ...].\"\n    #     )\n    if cu_seqlens is not None:\n        if q.shape[0] != 1:\n            raise ValueError(\n                f\"The batch size is expected to be 1 rather than {q.shape[0]} when using `cu_seqlens`.\"\n                f\"Please flatten variable-length inputs before processing.\"\n            )\n        if (\n            initial_state_indices is not None\n            and initial_state_indices.shape[0] != len(cu_seqlens) - 1\n        ):\n            raise ValueError(\n                f\"The number of initial states is expected to be equal to the number of input sequences, \"\n                f\"i.e., {len(cu_seqlens) - 1} rather than {initial_state_indices.shape[0]}.\"\n            )\n    if scale is None:\n        scale = k.shape[-1] ** -0.5\n    o, h = ChunkGatedDeltaRuleFunction.apply(\n        q,\n        k,\n        v,\n        g,","sourceCodeStart":216,"sourceCodeEnd":252,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/chunk.py#L216-L252","documentation":"chunk_gated_delta_rule supports variable-length sequences only via the cu_seqlens ragged format, which assumes all tokens are packed into a single batch row (batch size 1). If cu_seqlens is given and q.shape[0] != 1, inputs are not flattened and the kernel raises ValueError asking you to flatten them.","triggerScenarios":"Calling chunk_gated_delta_rule(q, ..., cu_seqlens=cu) where q is [B>1, T, H, ...] — i.e. passing a batched tensor together with cumulative-sequence-length indices.","commonSituations":"Migrating from per-sequence batching (padding) to varlen packing without reshaping; prefill paths feeding a [B, T, H] tensor while also computing cu_seqlens; forgetting q.reshape(1, -1, H) after concatenating sequences.","solutions":["Flatten q/k/v (and beta, g) to batch dim 1: q = q.reshape(1, -1, H) with tokens ordered to match cu_seqlens","Verify len(cu_seqlens) - 1 equals the number of packed sequences and initial_state_indices matches","Keep the padded-batch path and cu_seqlens mutually exclusive: use cu_seqlens only with flattened inputs"],"exampleFix":"# before\nout = chunk_gated_delta_rule(q, k, v, beta, g, cu_seqlens=cu)  # q: [4, T, H]\n# after\nq = q.reshape(1, -1, q.shape[-1])  # same for k, v, beta, g, ordered per cu_seqlens\nout = chunk_gated_delta_rule(q, k, v, beta, g, cu_seqlens=cu)","handlingStrategy":"validation","validationCode":"if cu_seqlens is not None:\n    assert q.shape[0] == 1, \"flatten variable-length inputs to [1, total_T, H] before using cu_seqlens\"\n    assert initial_state_indices is None or initial_state_indices.shape[0] == len(cu_seqlens) - 1","typeGuard":"def varlen_ready(q: torch.Tensor, cu_seqlens) -> bool:\n    return cu_seqlens is None or q.shape[0] == 1","tryCatchPattern":null,"preventionTips":["Always reshape packed varlen batches with q.reshape(1, -1, H) at the prefill entry point","Order tokens exactly per cu_seqlens before flattening"],"tags":["fla","gated-delta-rule","varlen","tensor-shape"],"backgroundTag":"shape-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}