sgl-project/sglang · error · ValueError

`initial_state` must be contiguous in the last dim.

Error message

`initial_state` must be contiguous in the last dim.

What it means

The Helion KDA decode kernel indexes the last dimension of initial_state with unit stride, so the tensor must be contiguous in that dim. validate_packed_decode_inputs checks initial_state.stride(-1) == 1 and raises when the state was sliced, transposed, or otherwise made non-contiguous in its innermost axis.

Source

Thrown at python/sglang/kernels/ops/attention/helion/kda_decode.py:285

    B = mixed_qkv.shape[0]
    if a.shape[0] != B or b.shape[0] != B:
        raise ValueError(
            "Mismatched batch sizes: "
            f"mixed_qkv.shape[0]={B}, a.shape[0]={a.shape[0]}, "
            f"b.shape[0]={b.shape[0]}."
        )
    if ssm_state_indices.shape[0] != B:
        raise ValueError(
            f"`ssm_state_indices` must have shape [B] "
            f"(got {tuple(ssm_state_indices.shape)}; expected ({B},))."
        )

    if initial_state.ndim != 4:
        raise ValueError(
            f"`initial_state` must be a 4D tensor (got ndim={initial_state.ndim})."
        )
    if initial_state.stride(-1) != 1:
        raise ValueError("`initial_state` must be contiguous in the last dim.")
    HV, V, K = initial_state.shape[-3:]
    if not _is_power_of_two(K) or not _is_power_of_two(V):
        raise ValueError(
            "Helion KDA decode requires power-of-two key and value head "
            f"dimensions (got K={K}, V={V})."
        )
    if a.shape[1] != HV * K:
        raise ValueError(
            f"`a` must have shape [B, HV*K] with HV={HV}, K={K} "
            f"(got a.shape={tuple(a.shape)})."
        )
    if b.shape[1] != HV:
        raise ValueError(
            f"`b` must have shape [B, HV] with HV={HV} (got b.shape={tuple(b.shape)})."
        )
    if A_log.numel() != HV:
        raise ValueError(f"`A_log` must have {HV} elements (got {A_log.numel()}).")
    if dt_bias.numel() != HV * K:

View on GitHub (pinned to 0132848349)

Solutions

1. Call initial_state = initial_state.contiguous() before the decode call
2. If the state comes from a pool view, rebuild the pool allocation so the last dim is contiguous
3. Avoid .t()/.transpose() on the state; store the pool in [.., V, K] layout directly

Example fix

// before
state = state_pool[indices].transpose(-1, -2)
out = helion_fused_recurrent_kda_packed_decode(..., state, ...)
// after
state = state_pool[indices].transpose(-1, -2).contiguous()
out = helion_fused_recurrent_kda_packed_decode(..., state, ...)

if initial_state.stride(-1) != 1:
    initial_state = initial_state.contiguous()

def last_dim_contiguous(t: torch.Tensor) -> bool:
    return t.stride(-1) == 1

When it happens

Trigger: Passing a state pool that was created via torch.transpose, narrow/slicing along the last dim, or expand of a smaller tensor into helion_fused_recurrent_kda_packed_decode / helion_fused_recurrent_kda_replayssm_decode.

Common situations: Reusing a state cache view created from a larger pool with a non-unit last-dim stride; converting weights from a checkpoint with a transposed layout; passing a broadcasted/expanded dummy state in tests.

Related errors

• `mixed_qkv` must be contiguous in the last dim.
• `a`/`b` must be contiguous in the last dim.
• `A_log`/`dt_bias` must be contiguous.
• `out` must be contiguous.
• f"recurrent_kda state inner strides must be compact (V*K, K,

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7fcc838de7c105ba. Report an issue: GitHub.