sgl-project/sglang · error · ValueError

`a`/`b` must be contiguous in the last dim.

Error message

`a`/`b` must be contiguous in the last dim.

What it means

The packed decode kernel requires unit stride in the last dimension of both gate tensors a and b. The wrapper checks a.stride(-1) != 1 or b.stride(-1) != 1 and raises when either is a non-contiguous view along the feature dim.

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_recurrent.py:291

    dt_bias: torch.Tensor,
    scale: float,
    initial_state: torch.Tensor,
    out: torch.Tensor,
    ssm_state_indices: torch.Tensor,
    use_qk_l2norm_in_kernel: bool = False,
) -> tuple[torch.Tensor, torch.Tensor]:
    if mixed_qkv.ndim != 2:
        raise ValueError(
            f"`mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim})."
        )
    if mixed_qkv.stride(-1) != 1:
        raise ValueError("`mixed_qkv` must be contiguous in the last dim.")
    if a.ndim != 2 or b.ndim != 2:
        raise ValueError(
            f"`a` and `b` must be 2D tensors (got a.ndim={a.ndim}, b.ndim={b.ndim})."
        )
    if a.stride(-1) != 1 or b.stride(-1) != 1:
        raise ValueError("`a`/`b` must be contiguous in the last dim.")
    if A_log.ndim != 1 or dt_bias.ndim != 1:
        raise ValueError("`A_log`/`dt_bias` must be 1D tensors.")
    if A_log.stride(0) != 1 or dt_bias.stride(0) != 1:
        raise ValueError("`A_log`/`dt_bias` must be contiguous.")
    if ssm_state_indices.ndim != 1:
        raise ValueError(
            f"`ssm_state_indices` must be 1D for packed decode (got ndim={ssm_state_indices.ndim})."
        )
    if not out.is_contiguous():
        raise ValueError("`out` must be contiguous.")

    dev = mixed_qkv.device
    if any(
        t.device != dev
        for t in (a, b, A_log, dt_bias, initial_state, out, ssm_state_indices)
    ):
        raise ValueError("All inputs must be on the same device.")

View on GitHub (pinned to 0132848349)

Solutions

1. Make both contiguous: a = a.contiguous(); b = b.contiguous()
2. Produce a and b with torch.chunk(2, dim=-1) on a (T, 2*HV) contiguous projection so both children inherit unit stride

Example fix

# before
a, b = g[..., 0], g[..., 1]  # stride-2 views
# after
a, b = g.unbind(dim=-1)
a = a.contiguous(); b = b.contiguous()

if a.stride(-1) != 1: a = a.contiguous()
if b.stride(-1) != 1: b = b.contiguous()

def gates_contiguous(a: torch.Tensor, b: torch.Tensor) -> bool:
    return a.stride(-1) == 1 and b.stride(-1) == 1

When it happens

Trigger: Creating a and b via stacking/chunking that leaves a stride > 1, e.g. a, b = gates[..., 0], gates[..., 1] on a (T, HV, 2) tensor (stride 2 in the last dim after transpose), or transposed views.

Common situations: Interleaved gate layouts where a and b alternate per element; using .unbind(-1) on a tensor whose last dim after permutation is not compact; caching gate tensors in a transposed buffer.

Related errors

• `mixed_qkv` must be contiguous in the last dim.
• `A_log`/`dt_bias` must be contiguous.
• `out` must be contiguous.
• `initial_state` must be contiguous in the last dim.
• `mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a6fa34c4710e5779. Report an issue: GitHub.