{"record":{"id":"692f6e9647251b83","repo":"sgl-project/sglang","slug":"the-number-of-intermediate-state-indices-is-expect","errorCode":null,"errorMessage":"The number of intermediate state indices is expected to be equal to the number of input sequences, i.e., {initial_state_indices.shape[0]} != {intermediate_state_indices.shape[0]}.","messagePattern":"The number of intermediate state indices is expected to be equal to the number of input sequences, i\\.e\\., (.+?) != (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent.py","lineNumber":1232,"sourceCode":"    intermediate_states_buffer: Optional[torch.Tensor] = None,\n    intermediate_state_indices: Optional[torch.Tensor] = None,\n    cache_steps: Optional[int] = None,\n    retrieve_parent_token: Optional[torch.Tensor] = None,\n) -> torch.Tensor:\n    if cu_seqlens is not None:\n        if q.shape[0] != 1:\n            raise ValueError(\n                f\"The batch size is expected to be 1 rather than {q.shape[0]} when using `cu_seqlens`.\"\n                f\"Please flatten variable-length inputs before processing.\"\n            )\n        if initial_state_source is not None:\n            if initial_state_indices.shape[0] != len(cu_seqlens) - 1:\n                raise ValueError(\n                    f\"The number of initial states is expected to be equal to the number of input sequences, \"\n                    f\"i.e., {len(cu_seqlens) - 1} rather than {initial_state_indices.shape[0]}.\"\n                )\n            if initial_state_indices.shape[0] != intermediate_state_indices.shape[0]:\n                raise ValueError(\n                    f\"The number of intermediate state indices is expected to be equal to the number of input sequences, \"\n                    f\"i.e., {initial_state_indices.shape[0]} != {intermediate_state_indices.shape[0]}.\"\n                )\n    if scale is None:\n        scale = k.shape[-1] ** -0.5\n    else:\n        assert scale > 0, \"scale must be positive\"\n    if beta is None:\n        beta = torch.ones_like(q[..., 0])\n    o = FusedRecurrentUpdateFunction.apply(\n        q,\n        k,\n        v,\n        g,\n        beta,\n        scale,\n        initial_state_source,\n        initial_state_indices,","sourceCodeStart":1214,"sourceCodeEnd":1250,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent.py#L1214-L1250","documentation":"When both initial_state_indices and intermediate_state_indices are passed to fused_recurrent_gated_delta_rule_update with cu_seqlens, they must have identical lengths (one per sequence). This check fires when the intermediate (output) state slot tensor doesn't line up with the input state slots.","triggerScenarios":"Allocating the output/intermediate state index tensor with a different batch length than the input index tensor, e.g. one per token instead of per sequence.","commonSituations":"Reworking a state cache: input slots sized per-sequence but flush slots sized per-token or per-head; partial updates after sequences finish mid-batch.","solutions":["Allocate both index tensors with shape [len(cu_seqlens)-1]","When flushing per-token states, group them per sequence or use the appropriate API variant","Log both shapes before the call during development"],"exampleFix":"// before\ninitial_state_indices = torch.arange(N)  # [N]\nintermediate_state_indices = torch.arange(N*T)  # [N*T] -> mismatch\n// after\ninitial_state_indices = torch.arange(N)\nintermediate_state_indices = flush_slots[:N]  # [N], one per sequence","handlingStrategy":"validation","validationCode":"assert initial_state_indices.shape == intermediate_state_indices.shape, 'slot tensors must match'","typeGuard":"def valid_slot_pair(init_idx, inter_idx) -> bool:\n    return init_idx.ndim == 1 and init_idx.shape == inter_idx.shape","tryCatchPattern":null,"preventionTips":["Allocate input and flush slot tensors together from one shape","Add a debug assert in the serving loop"],"tags":["pytorch","state-management","gdn","validation"],"backgroundTag":"cu-seqlens-count-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}