{"record":{"id":"3e3f60987190a4ed","repo":"sgl-project/sglang","slug":"initial-state-must-be-4d-got-ndim-initial-stat","errorCode":null,"errorMessage":"`initial_state` must be 4D (got ndim={initial_state.ndim}).","messagePattern":"`initial_state` must be 4D \\(got ndim=(.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent_linear_replayssm.py","lineNumber":486,"sourceCode":"    (``d_cache`` / ``k_cache`` / ``g_cache``) and the per-decode-row\n    ``write_pos`` cursor.  ``initial_state`` is both the checkpoint read (h0)\n    and the (flush-only) checkpoint write (ht), in place.\n\n    Allocates nothing persistent: the caller owns the ring tensors and is\n    responsible for advancing / resetting ``write_pos`` (e.g. ``(write_pos+1) %\n    L`` after each step).  This is a STANDALONE kernel; the memory-pool / cache\n    integration is a later phase.\n    \"\"\"\n    if mixed_qkv.ndim != 2:\n        raise ValueError(f\"`mixed_qkv` must be 2D (got ndim={mixed_qkv.ndim}).\")\n    if mixed_qkv.stride(-1) != 1:\n        raise ValueError(\"`mixed_qkv` must be contiguous in the last dim.\")\n    if b.ndim != 2:\n        raise ValueError(f\"`b` must be 2D (got b.ndim={b.ndim}).\")\n    if A_log.ndim != 1:\n        raise ValueError(\"`A_log` must be a 1D tensor.\")\n    if initial_state.ndim != 4:\n        raise ValueError(f\"`initial_state` must be 4D (got ndim={initial_state.ndim}).\")\n    if not out.is_contiguous():\n        raise ValueError(\"`out` must be contiguous.\")\n    if write_pos.ndim != 1 or write_pos.dtype != torch.int32:\n        raise ValueError(\"`write_pos` must be a 1D int32 tensor.\")\n    if force_flush is not None and (\n        force_flush.ndim != 1 or force_flush.dtype != torch.int32\n    ):\n        raise ValueError(\"`force_flush` must be a 1D int32 tensor or None.\")\n\n    B = mixed_qkv.shape[0]\n    num_state_slots, HV, V, K = initial_state.shape\n    qkv_dim = mixed_qkv.shape[1]\n    q_dim = (qkv_dim - HV * V) // 2\n    if q_dim <= 0 or q_dim % K != 0:\n        raise ValueError(\n            f\"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}, K={K}.\"\n        )\n    H = q_dim // K","sourceCodeStart":468,"sourceCodeEnd":504,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent_linear_replayssm.py#L468-L504","documentation":"The recurrent state for replaySSM decode must be 4D, typically [N, HV, K, V] (slots, heads, key dim, value dim). Passing a 3D or 5D state tensor means the state layout doesn't match the kernel's expectation and the update would read/write out of bounds.","triggerScenarios":"Passing a flattened state pool [N, HV*K*V], or a per-batch state [B, HV, K*V] with wrong rank.","commonSituations":"Integrating with a memory pool that stores states as flat rows; converting between chunked-kernel state layout [B, H, K, V] and a slot-indexed pool with extra dims.","solutions":["Reshape the state to 4D: state.view(num_slots, HV, K, V) (keep it contiguous)","Verify the pool's per-slot stride equals HV*K*V elements","Check ring/write_pos tensors are 1D int32 to satisfy sibling checks"],"exampleFix":"// before\nstate = pool_flat[slots]  # [N, HV*K*V]\n// after\nstate = pool_flat.view(num_slots, HV, K, V)[slots]  # 4D view","handlingStrategy":"validation","validationCode":"assert initial_state.ndim == 4, initial_state.shape\ninitial_state = initial_state.view(-1, HV, K, V) if initial_state.ndim != 4 else initial_state","typeGuard":"def is_4d_state(s: torch.Tensor) -> bool:\n    return s.ndim == 4","tryCatchPattern":null,"preventionTips":["Store state pools as [slots, HV, K, V] tensors","Never flatten state rows; index the 4D pool directly"],"tags":["pytorch","tensor-shape","replayssm","state-management"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}