sgl-project/sglang · error · ValueError

KDA `a` must be a contiguous 2D or 3D tensor.

Error message

KDA `a` must be a contiguous 2D or 3D tensor.

What it means

helion_fused_recurrent_kda_replayssm_decode requires the KDA decay input `a` to be 2D or 3D AND contiguous, because it is flattened with a.view(batch, -1) — a view, not a reshape — before being passed to validate_packed_decode_inputs. Non-contiguous or higher-rank tensors fail this precondition.

Source

Thrown at python/sglang/kernels/ops/attention/helion/kda_replayssm.py:707

    g_cache: torch.Tensor,
    out: torch.Tensor,
    ssm_state_indices: torch.Tensor,
    write_pos: torch.Tensor,
    force_flush: torch.Tensor | None = None,
    use_qk_l2norm_in_kernel: bool = False,
    lower_bound: float | None = None,
) -> tuple[torch.Tensor, torch.Tensor]:
    """Run one buffered KDA decode step using caller-owned ReplaySSM state.

    Allocates nothing persistent: the caller owns ``d_cache`` / ``k_cache`` /
    ``g_cache`` and is responsible for advancing ``write_pos`` modulo the ring
    length after a non-flush step and resetting it to zero after a natural or
    forced flush. ``initial_state`` is both the checkpoint read (h0) and the
    flush-only checkpoint write (ht), in place.
    """
    batch = mixed_qkv.size(0)
    if a.ndim not in (2, 3) or not a.is_contiguous():
        raise ValueError("KDA `a` must be a contiguous 2D or 3D tensor.")
    if dt_bias.ndim not in (1, 2) or not dt_bias.is_contiguous():
        raise ValueError("KDA `dt_bias` must be a contiguous 1D or 2D tensor.")
    flat_a = a.view(batch, -1)
    flat_dt_bias = dt_bias.view(-1)
    _, num_q_heads, num_v_heads, key_dim, value_dim = validate_packed_decode_inputs(
        mixed_qkv,
        flat_a,
        b,
        A_log,
        flat_dt_bias,
        initial_state,
        out,
        ssm_state_indices,
    )

    if write_pos.ndim != 1 or write_pos.dtype is not torch.int32:
        raise ValueError("`write_pos` must be a 1D int32 tensor.")
    if write_pos.shape != (batch,):

View on GitHub (pinned to 0132848349)

Solutions

1. Reshape and make contiguous: a = a.reshape(batch, -1).contiguous()
2. If a is [B, T, HV, K] from a multi-step path, select the current step a[:, t] first
3. Avoid passing views with holes — materialize with .contiguous() once at the producer

Example fix

// before
a = a.transpose(1, 2)  # non-contiguous
helion_fused_recurrent_kda_replayssm_decode(..., a=a, ...)
// after
a = a.transpose(1, 2).contiguous()
helion_fused_recurrent_kda_replayssm_decode(..., a=a, ...)

if a.ndim not in (2, 3) or not a.is_contiguous():
    a = a.reshape(mixed_qkv.size(0), -1).contiguous()

def valid_replay_a(a: torch.Tensor) -> bool:
    return a.ndim in (2, 3) and a.is_contiguous()

When it happens

Trigger: Passing `a` that is 4D, non-contiguous (from transpose/ slicing), or a lazily-expanded tensor to the replayssm decode; a.view(batch, -1) would throw for such inputs, so the guard fires first with a clearer message.

Common situations: Feeding the raw conv/gate projection output without reshaping; passing a transposed tensor from a NHWC-style layout; test tensors created via torch.randn(...).transpose(0, 1).

Related errors

• KDA `dt_bias` must be a contiguous 1D or 2D tensor.
• `write_pos` must be a 1D int32 tensor.
• `mixed_qkv` must be contiguous in the last dim.
• `out` must be contiguous.
• `mixed_qkv` must be contiguous in the last dim.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/08daeb5be57afde3. Report an issue: GitHub.