{"record":{"id":"39f78f9c15a7c54b","repo":"sgl-project/sglang","slug":"the-batch-size-is-expected-to-be-1-rather-than-q-39f78f","errorCode":null,"errorMessage":"The batch size is expected to be 1 rather than {q.shape[0]} when using `cu_seqlens`.Please flatten variable-length inputs before processing.","messagePattern":"The batch size is expected to be 1 rather than (.+?) when using `cu_seqlens`\\.Please flatten variable-length inputs before processing\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/kda.py","lineNumber":154,"sourceCode":"    return o, final_state\n\n\ndef fused_recurrent_kda(\n    q: torch.Tensor,\n    k: torch.Tensor,\n    v: torch.Tensor,\n    g: torch.Tensor,\n    beta: torch.Tensor = None,\n    scale: float = None,\n    initial_state: torch.Tensor = None,\n    inplace_final_state: bool = True,\n    use_qk_l2norm_in_kernel: bool = True,\n    cu_seqlens: torch.LongTensor | None = None,\n    # ssm_state_indices: torch.LongTensor | None = None,\n    **kwargs,\n) -> tuple[torch.Tensor, torch.Tensor]:\n    if cu_seqlens is not None and q.shape[0] != 1:\n        raise ValueError(\n            f\"The batch size is expected to be 1 rather than {q.shape[0]} when using `cu_seqlens`.\"\n            f\"Please flatten variable-length inputs before processing.\"\n        )\n    if scale is None:\n        scale = k.shape[-1] ** -0.5\n\n    o, final_state = fused_recurrent_kda_fwd(\n        q=q.contiguous(),\n        k=k.contiguous(),\n        v=v.contiguous(),\n        g=g.contiguous(),\n        beta=beta.contiguous(),\n        scale=scale,\n        initial_state=initial_state,\n        inplace_final_state=inplace_final_state,\n        cu_seqlens=cu_seqlens,\n        # ssm_state_indices=ssm_state_indices,\n        use_qk_l2norm_in_kernel=use_qk_l2norm_in_kernel,","sourceCodeStart":136,"sourceCodeEnd":172,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/kda.py#L136-L172","documentation":"fused_recurrent_kda in the KDA (Kimi Delta Attention) FLA kernel requires that when cu_seqlens (variable-length cumulative sequence lengths) is passed, the q tensor's batch dimension must be 1. Variable-length sequences must be flattened (packed) into a single [1, total_tokens, ...] tensor with cu_seqlens describing boundaries.","triggerScenarios":"Calling fused_recurrent_kda(q, k, ...) with cu_seqlens set and q.shape[0] != 1, i.e. passing a batched [B, T, ...] tensor instead of a flattened varlen tensor.","commonSituations":"Batching multiple sequences of different lengths into a padded [B, T, ...] tensor and passing cu_seqlens alongside; migrating from chunked KDA kernels that accept batched inputs.","solutions":["Flatten q/k/v to shape [1, total_tokens, ...] and pass cu_seqlens describing per-sequence boundaries","Ensure batch dims are collapsed: q.reshape(1, -1, q.shape[-1])","If you need per-sequence states, use ssm_state_indices / loop over sequences instead of a batch dim"],"exampleFix":"// before\nq = q  # [B, T, D] with cu_seqlens set\n// after\nq = q.reshape(1, -1, q.shape[-1])\nk = k.reshape(1, -1, k.shape[-1])\nv = v.reshape(1, -1, v.shape[-1])\nout, _ = fused_recurrent_kda(q, k, v, cu_seqlens=cu_seqlens)","handlingStrategy":"validation","validationCode":"assert cu_seqlens is None or q.shape[0] == 1, 'flatten varlen inputs to [1, total_T, D] before fused_recurrent_kda'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always pack variable-length batches with q.reshape(1, -1, D) and cu_seqlens","Write a small packing helper so all call sites flatten consistently"],"tags":["kda","fla","varlen","batch-shape","attention"],"backgroundTag":"varlen-input-flattening-required","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}