sgl-project/sglang · error · ValueError

`mixed_qkv` must be contiguous in the last dim.

Error message

`mixed_qkv` must be contiguous in the last dim.

What it means

The packed decode Triton kernel indexes mixed_qkv assuming unit stride in the last dimension. The Python wrapper checks mixed_qkv.stride(-1) != 1 and raises when the qkv rows are not contiguous in the feature dimension (e.g. a sliced or transposed view).

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_recurrent.py:285

def fused_recurrent_gated_delta_rule_packed_decode(
    mixed_qkv: torch.Tensor,
    a: torch.Tensor,
    b: torch.Tensor,
    A_log: torch.Tensor,
    dt_bias: torch.Tensor,
    scale: float,
    initial_state: torch.Tensor,
    out: torch.Tensor,
    ssm_state_indices: torch.Tensor,
    use_qk_l2norm_in_kernel: bool = False,
) -> tuple[torch.Tensor, torch.Tensor]:
    if mixed_qkv.ndim != 2:
        raise ValueError(
            f"`mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim})."
        )
    if mixed_qkv.stride(-1) != 1:
        raise ValueError("`mixed_qkv` must be contiguous in the last dim.")
    if a.ndim != 2 or b.ndim != 2:
        raise ValueError(
            f"`a` and `b` must be 2D tensors (got a.ndim={a.ndim}, b.ndim={b.ndim})."
        )
    if a.stride(-1) != 1 or b.stride(-1) != 1:
        raise ValueError("`a`/`b` must be contiguous in the last dim.")
    if A_log.ndim != 1 or dt_bias.ndim != 1:
        raise ValueError("`A_log`/`dt_bias` must be 1D tensors.")
    if A_log.stride(0) != 1 or dt_bias.stride(0) != 1:
        raise ValueError("`A_log`/`dt_bias` must be contiguous.")
    if ssm_state_indices.ndim != 1:
        raise ValueError(
            f"`ssm_state_indices` must be 1D for packed decode (got ndim={ssm_state_indices.ndim})."
        )
    if not out.is_contiguous():
        raise ValueError("`out` must be contiguous.")

    dev = mixed_qkv.device

View on GitHub (pinned to 0132848349)

Solutions

1. Call .contiguous() on mixed_qkv before the kernel (or ensure it comes directly from a contiguous Linear output)
2. Restructure upstream slicing to produce compact rows: mixed_qkv = buf[:, :qkv_dim].contiguous()

Example fix

# before
mixed_qkv = mixed_qkv_proj[:, :qkv_dim]  # row stride > qkv_dim
# after
mixed_qkv = mixed_qkv_proj[:, :qkv_dim].contiguous()

mixed_qkv = mixed_qkv.contiguous() if mixed_qkv.stride(-1) != 1 else mixed_qkv

def last_dim_contiguous(t: torch.Tensor) -> bool:
    return t.stride(-1) == 1

When it happens

Trigger: Passing mixed_qkv created via transpose, narrow/slicing of the last dim, or expand; e.g. mixed_qkv = proj[..., ::2] or a (T, D) slice of a wider buffer with row stride > D.

Common situations: Slicing a fused qkv-plus-gates projection buffer and passing a non-compacted view; reusing a cached projection tensor after a .transpose(0, 1); tensors produced by torch.chunk on dim 1 followed by no .contiguous() where strides leak.

Related errors

• `a`/`b` must be contiguous in the last dim.
• `A_log`/`dt_bias` must be contiguous.
• `out` must be contiguous.
• `initial_state` must be contiguous in the last dim.
• `mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/43b1bc996af97dd2. Report an issue: GitHub.