sgl-project/sglang · error · ValueError

f"recurrent_kda state inner strides must be compact (V*K, K,

Error message

f"recurrent_kda state inner strides must be compact (V*K, K, 1)=({v * k}, {k}, 1); got {ssm_states.stride()[1:]} (only the slot stride may be non-compact)"

What it means

Beyond shape, recurrent_kda requires the inner (HV, V, K) dims of the state pool to be compactly strided (V*K, K, 1). Only the slot (first) dimension stride may be padded. Non-contiguous inner strides would silently corrupt in-kernel state, so it fails loudly.

Source

Thrown at python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py:111

        and ``assumed_align=32`` (flashinfer ``kda_kernels/recurrent_kda.py``):
        the slot stride is free — which is what lets the envelope-strided pools
        (unified memory / page-major layout, slot stride = per-slot envelope
        pitch) be passed in and updated IN PLACE on the cu_seqlens path — but
        the inner strides are compiled-in constants and the divisibility /
        alignment are hard assumptions. A pool violating them would mis-address
        state in-kernel without any error; fail loudly here instead.
        """
        key = id(ssm_states)
        if key in self._state_contract_ok:
            return
        if ssm_states.dim() != 4:
            raise ValueError(
                f"recurrent_kda needs a [N, HV, V, K] state pool; got "
                f"shape {tuple(ssm_states.shape)}"
            )
        _, hv, v, k = ssm_states.shape
        if ssm_states.stride()[1:] != (v * k, k, 1):
            raise ValueError(
                "recurrent_kda state inner strides must be compact "
                f"(V*K, K, 1)=({v * k}, {k}, 1); got {ssm_states.stride()[1:]} "
                "(only the slot stride may be non-compact)"
            )
        base_bytes = ssm_states.storage_offset() * ssm_states.element_size()
        if ssm_states.stride(0) % 16 != 0 or base_bytes % 32 != 0:
            raise ValueError(
                "recurrent_kda state pool breaks the compiled stride contract: "
                f"slot stride {ssm_states.stride(0)} elements must be a multiple "
                f"of 16 and the base byte offset {base_bytes} a multiple of 32 "
                "(sym_int64(divisibility=16) / assumed_align=32)"
            )
        self._state_contract_ok.add(key)

    # ---- gate / beta normalization (shared by decode + verify) ----

    def _prep_gate_params(self, A_log: torch.Tensor, dt_bias: torch.Tensor):
        # A_log: [1, 1, H, 1] -> [H] fp32; dt_bias: [H*K] (1D) -> fp32. Cached per

View on GitHub (pinned to 0132848349)

Solutions

1. Materialize the pool as [N, HV, V, K] and call .contiguous() so inner strides are (V*K, K, 1)
2. If converting from a vendor K-major layout, transpose then contiguous() before decode
3. Keep padding only on the slot (dim 0) stride

Example fix

# before
ssm_states = vendor_state  # [N, HV, K, V] strides
# after
ssm_states = vendor_state.transpose(-1, -2).contiguous()  # [N, HV, V, K] compact

_, hv, v, k = ssm_states.shape
assert tuple(ssm_states.stride()[1:]) == (v * k, k, 1), f'non-compact inner strides: {ssm_states.stride()}'

def has_compact_inner_strides(t: torch.Tensor) -> bool:
    if t.dim() != 4: return False
    _, _, v, k = t.shape
    return tuple(t.stride()[1:]) == (v * k, k, 1)

When it happens

Trigger: Passing a state pool that is a transpose/slice with non-unit inner stride, e.g. state.transpose(-1,-2) of the SGLang layout, or a view over a KV-interleaved buffer.

Common situations: Custom kernels or tests that materialize states in the NVIDIA [B,H,K,V] layout and pass it directly to FlashInfer decode without a .contiguous() round-trip.

Related errors

• `initial_state` must be contiguous in the last dim.
• f"recurrent_kda needs a [N, HV, V, K] state pool; got shape
• v_cache must be provided
• q can only be None when only_qv=True
• q must be provided unless qv is provided with only_qv=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/abc9cf5de295bcef. Report an issue: GitHub.