sgl-project/sglang · error · ValueError

out must have stride 1 in the last dimension

Error message

out must have stride 1 in the last dimension

What it means

The FA4 kernel writes output via TMA/vectorized stores that require the last dimension of the user-supplied out tensor to be contiguous (stride 1). A strided last dim would cause incorrect or illegal memory access.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/interface.py:646

        lse_shape = (
            (batch_size, seqlen_q, num_head)
            if cu_seqlens_q is None
            else (total_q, num_head)
        )

    if out is None:
        out = torch.empty(
            *q_batch_seqlen_shape,
            num_head,
            head_dim_v,
            dtype=out_torch_dtype,
            device=device,
        )
    else:
        if out.requires_grad:
            raise ValueError("out must not require gradients")
        if out.stride(-1) != 1:
            raise ValueError("out must have stride 1 in the last dimension")
        _validate_tensor(
            out,
            "out",
            (*q_batch_seqlen_shape, num_head, head_dim_v),
            out_torch_dtype,
            device,
        )

    if lse is None:
        lse = (
            torch.empty(lse_shape, dtype=torch.float32, device=device)
            if requires_grad or return_lse
            else None
        )
    elif lse is not None:
        _validate_tensor(lse, "lse", lse_shape, torch.float32, device)

    if seqlen_k == 0 or total_q == 0:

View on GitHub (pinned to 0132848349)

Solutions

1. Pass a contiguous output: out = out.contiguous() before the call
2. Or allocate out fresh with torch.empty(expected_shape, dtype, device)
3. If writing into a bigger buffer, copy back afterwards instead of passing a strided view

Example fix

// before
fa(..., out=buf[:, :, ::2])
// after
out = torch.empty(shape, dtype=dt, device=dev)
fa(..., out=out)
buf[:, :, ::2] = out

if out is not None and out.stride(-1) != 1:
    out = out.contiguous()

def out_stride_safe(out) -> bool: return out is None or out.stride(-1) == 1

When it happens

Trigger: Passing out= whose last-dimension stride != 1, typically a slice/transpose/view (e.g. out = buf[:, :, ::2] or out = buf.transpose(-1,-2)).

Common situations: Writing into a preallocated buffer viewed with padding or channels-last-like layouts; slicing a larger workspace tensor.

Related errors

• out must have stride 1 in the last dimension
• out must not require gradients
• out must not require gradients
• The layout of q is not supported
• The layout of k is not supported

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8537ef62b06697f7. Report an issue: GitHub.