{"record":{"id":"eb65ae4ab1115f33","repo":"sgl-project/sglang","slug":"the-number-of-initial-states-is-expected-to-be-equ-eb65ae","errorCode":null,"errorMessage":"The number of initial states is expected to be equal to the number of input sequences, i.e., {len(cu_seqlens) - 1} rather than {initial_state_indices.shape[0]}.","messagePattern":"The number of initial states is expected to be equal to the number of input sequences, i\\.e\\., (.+?) rather than (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent.py","lineNumber":1227,"sourceCode":"    initial_state_indices: torch.Tensor = None,\n    cu_seqlens: Optional[torch.LongTensor] = None,\n    use_qk_l2norm_in_kernel: bool = False,\n    disable_state_update: bool = False,\n    disable_output_calculation: bool = False,\n    intermediate_states_buffer: Optional[torch.Tensor] = None,\n    intermediate_state_indices: Optional[torch.Tensor] = None,\n    cache_steps: Optional[int] = None,\n    retrieve_parent_token: Optional[torch.Tensor] = None,\n) -> torch.Tensor:\n    if cu_seqlens is not None:\n        if q.shape[0] != 1:\n            raise ValueError(\n                f\"The batch size is expected to be 1 rather than {q.shape[0]} when using `cu_seqlens`.\"\n                f\"Please flatten variable-length inputs before processing.\"\n            )\n        if initial_state_source is not None:\n            if initial_state_indices.shape[0] != len(cu_seqlens) - 1:\n                raise ValueError(\n                    f\"The number of initial states is expected to be equal to the number of input sequences, \"\n                    f\"i.e., {len(cu_seqlens) - 1} rather than {initial_state_indices.shape[0]}.\"\n                )\n            if initial_state_indices.shape[0] != intermediate_state_indices.shape[0]:\n                raise ValueError(\n                    f\"The number of intermediate state indices is expected to be equal to the number of input sequences, \"\n                    f\"i.e., {initial_state_indices.shape[0]} != {intermediate_state_indices.shape[0]}.\"\n                )\n    if scale is None:\n        scale = k.shape[-1] ** -0.5\n    else:\n        assert scale > 0, \"scale must be positive\"\n    if beta is None:\n        beta = torch.ones_like(q[..., 0])\n    o = FusedRecurrentUpdateFunction.apply(\n        q,\n        k,\n        v,","sourceCodeStart":1209,"sourceCodeEnd":1245,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent.py#L1209-L1245","documentation":"In fused_recurrent_gated_delta_rule_update, when initial_state_source is provided with cu_seqlens, initial_state_indices must have exactly one row per input sequence, i.e. len(cu_seqlens)-1 entries. The mismatch means the state-index tensor doesn't cover every sequence in the packed batch.","triggerScenarios":"Passing initial_state_indices with batch size different from the number of var-len segments (e.g. reusing a per-request buffer sized from a previous batch).","commonSituations":"Serving with dynamic batch sizes where the index tensor is a persistent pool buffer not resized per step; off-by-one when building cu_seqlens.","solutions":["Resize/refresh initial_state_indices to length len(cu_seqlens)-1 each step","Verify cu_seqlens has num_seqs+1 entries starting at 0 and ending at total_T","Index into the state pool per sequence before the call"],"exampleFix":"// before\ninitial_state_indices = pool_idx  # stale length from previous batch\n// after\ninitial_state_indices = torch.arange(num_seqs, dtype=torch.int32, device=dev)  # len == len(cu_seqlens)-1","handlingStrategy":"validation","validationCode":"n = len(cu_seqlens) - 1\nassert initial_state_indices.shape[0] == n\nassert initial_state_source.shape[0] >= n","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Recompute state index tensors every scheduler step","Validate cu_seqlens starts at 0 and ends at total_T"],"tags":["pytorch","variable-length","state-management","gdn","validation"],"backgroundTag":"cu-seqlens-count-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}