sgl-project/sglang · error · ValueError

KDA `dt_bias` must be a contiguous 1D or 2D tensor.

Error message

KDA `dt_bias` must be a contiguous 1D or 2D tensor.

What it means

The replayssm decode flattens dt_bias with dt_bias.view(-1) before validation, so dt_bias must be 1D or 2D and contiguous. Anything else (3D+, non-contiguous, or an expanded tensor) makes the view illegal, and this guard raises a descriptive error before that happens.

Source

Thrown at python/sglang/kernels/ops/attention/helion/kda_replayssm.py:709

    ssm_state_indices: torch.Tensor,
    write_pos: torch.Tensor,
    force_flush: torch.Tensor | None = None,
    use_qk_l2norm_in_kernel: bool = False,
    lower_bound: float | None = None,
) -> tuple[torch.Tensor, torch.Tensor]:
    """Run one buffered KDA decode step using caller-owned ReplaySSM state.

    Allocates nothing persistent: the caller owns ``d_cache`` / ``k_cache`` /
    ``g_cache`` and is responsible for advancing ``write_pos`` modulo the ring
    length after a non-flush step and resetting it to zero after a natural or
    forced flush. ``initial_state`` is both the checkpoint read (h0) and the
    flush-only checkpoint write (ht), in place.
    """
    batch = mixed_qkv.size(0)
    if a.ndim not in (2, 3) or not a.is_contiguous():
        raise ValueError("KDA `a` must be a contiguous 2D or 3D tensor.")
    if dt_bias.ndim not in (1, 2) or not dt_bias.is_contiguous():
        raise ValueError("KDA `dt_bias` must be a contiguous 1D or 2D tensor.")
    flat_a = a.view(batch, -1)
    flat_dt_bias = dt_bias.view(-1)
    _, num_q_heads, num_v_heads, key_dim, value_dim = validate_packed_decode_inputs(
        mixed_qkv,
        flat_a,
        b,
        A_log,
        flat_dt_bias,
        initial_state,
        out,
        ssm_state_indices,
    )

    if write_pos.ndim != 1 or write_pos.dtype is not torch.int32:
        raise ValueError("`write_pos` must be a 1D int32 tensor.")
    if write_pos.shape != (batch,):
        raise ValueError(f"`write_pos` must have shape {(batch,)}.")
    if force_flush is not None and (

View on GitHub (pinned to 0132848349)

Solutions

1. Flatten and make contiguous: dt_bias = dt_bias.reshape(-1).contiguous()
2. Load dt_bias as [HV, K] and pass it directly, or squeeze() stray singleton dims
3. Verify numel == HV*K after flattening (the downstream validate will check)

Example fix

// before
dt_bias = layer.dt_bias.unsqueeze(0).expand(T, -1, -1)  # 3D, expanded
// after
dt_bias = layer.dt_bias.reshape(-1).contiguous()  # [HV*K]

if dt_bias.ndim not in (1, 2) or not dt_bias.is_contiguous():
    dt_bias = dt_bias.reshape(-1).contiguous()

def valid_replay_dt_bias(t: torch.Tensor) -> bool:
    return t.ndim in (1, 2) and t.is_contiguous()

When it happens

Trigger: Passing dt_bias stored as [HV, K] but made non-contiguous via slicing/transpose, or a 3D [1, HV, K] tensor, into helion_fused_recurrent_kda_replayssm_decode.

Common situations: Loading dt_bias from a checkpoint and slicing off a leading batch dim incorrectly; broadcasting dt_bias across time steps with expand; reusing a test helper that returns a non-contiguous tensor.

Related errors

• KDA `a` must be a contiguous 2D or 3D tensor.
• `mixed_qkv` must be contiguous in the last dim.
• `out` must be contiguous.
• `mixed_qkv` must be contiguous in the last dim.
• `a`/`b` must be contiguous in the last dim.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c859d5d34d7a716a. Report an issue: GitHub.