sgl-project/sglang · error · ValueError

`out` must be contiguous.

Error message

`out` must be contiguous.

What it means

The output tensor out must be contiguous because the Triton kernel writes results with compact indexing. The wrapper checks out.is_contiguous() and raises when a preallocated output buffer is a strided view (e.g. a slice of a larger circular cache or a transposed buffer).

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_recurrent.py:301

        )
    if mixed_qkv.stride(-1) != 1:
        raise ValueError("`mixed_qkv` must be contiguous in the last dim.")
    if a.ndim != 2 or b.ndim != 2:
        raise ValueError(
            f"`a` and `b` must be 2D tensors (got a.ndim={a.ndim}, b.ndim={b.ndim})."
        )
    if a.stride(-1) != 1 or b.stride(-1) != 1:
        raise ValueError("`a`/`b` must be contiguous in the last dim.")
    if A_log.ndim != 1 or dt_bias.ndim != 1:
        raise ValueError("`A_log`/`dt_bias` must be 1D tensors.")
    if A_log.stride(0) != 1 or dt_bias.stride(0) != 1:
        raise ValueError("`A_log`/`dt_bias` must be contiguous.")
    if ssm_state_indices.ndim != 1:
        raise ValueError(
            f"`ssm_state_indices` must be 1D for packed decode (got ndim={ssm_state_indices.ndim})."
        )
    if not out.is_contiguous():
        raise ValueError("`out` must be contiguous.")

    dev = mixed_qkv.device
    if any(
        t.device != dev
        for t in (a, b, A_log, dt_bias, initial_state, out, ssm_state_indices)
    ):
        raise ValueError("All inputs must be on the same device.")

    B = mixed_qkv.shape[0]
    if a.shape[0] != B or b.shape[0] != B:
        raise ValueError(
            "Mismatched batch sizes: "
            f"mixed_qkv.shape[0]={B}, a.shape[0]={a.shape[0]}, b.shape[0]={b.shape[0]}."
        )
    if ssm_state_indices.shape[0] != B:
        raise ValueError(
            f"`ssm_state_indices` must have shape [B] (got {tuple(ssm_state_indices.shape)}; expected ({B},))."
        )

View on GitHub (pinned to 0132848349)

Solutions

  1. Allocate out with torch.empty((B, 1, HV, V), dtype=..., device=...) (naturally contiguous) and copy into the cache afterwards, or pass a .contiguous() view
  2. If writing into a cache is required, do out_cache[slots] = out after the kernel

Example fix

# before
out = out_buf[:, :, t]  # strided view
# after
out = torch.empty((B, 1, HV, V), dtype=x.dtype, device=x.device)
out, state = fused_recurrent_gated_delta_rule_packed_decode(..., out=out, ...)
out_buf[:, :, t] = out
Defensive patterns

Strategy: validation

Validate before calling

if not out.is_contiguous():
    out = torch.empty_like(out, memory_format=torch.contiguous_format)

Type guard

def contiguous_out(o: torch.Tensor) -> bool:
    return o.is_contiguous()

Prevention

When it happens

Trigger: Passing out = out_cache[locs] views, out = buf[:, :, t, :] style slices, or outputs created with as_strided/slice of paged buffers.

Common situations: Zero-copy output into a paged/ring output cache; DP/TP code reusing a padded output buffer; test harnesses allocating out with unusual strides via torch.empty followed by slicing.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4c83c60bb2a16eef. Report an issue: GitHub.