sgl-project/sglang · error · ValueError

out must have stride 1 in the last dimension

Error message

out must have stride 1 in the last dimension

What it means

FA4 sm120 kernels require the out tensor's last dimension to be contiguous (stride 1) for coalesced vectorized stores; any other last-dim stride raises ValueError in _validate_out_contract.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attention_v4_sm120.py:73

        resolve_runtime_policy(
            device_capability=device_capability,
            deterministic=deterministic,
        )
    )
    return FlashAttentionV4SM120RuntimePolicy(
        num_splits=num_splits,
        decode_num_splits=decode_num_splits,
        decode_uses_static_max_seqlen_k=decode_uses_static_max_seqlen_k,
    )


def _validate_out_contract(out: Optional[torch.Tensor]) -> None:
    if out is None:
        return
    if out.requires_grad:
        raise ValueError("out must not require gradients")
    if out.stride(-1) != 1:
        raise ValueError("out must have stride 1 in the last dimension")


@debug_kernel_api
def flash_attn_varlen_func(
    q: torch.Tensor,
    k: torch.Tensor,
    v: torch.Tensor,
    cu_seqlens_q: Optional[torch.Tensor] = None,
    cu_seqlens_k: Optional[torch.Tensor] = None,
    qv: Optional[torch.Tensor] = None,
    seqused_q: Optional[torch.Tensor] = None,
    seqused_k: Optional[torch.Tensor] = None,
    max_seqlen_q: Optional[int] = None,
    max_seqlen_k: Optional[int] = None,
    page_table: Optional[torch.Tensor] = None,
    softmax_scale: Optional[float] = None,
    causal: bool = False,
    softcap: Optional[float] = None,

View on GitHub (pinned to 0132848349)

Solutions

1. Make out contiguous: out = out.contiguous()
2. Allocate a fresh contiguous buffer: out = torch.empty_like(q)
3. Restructure so the last dim is unit-stride before calling

Example fix

# before
out = buf.transpose(1, 2)  # last-dim stride != 1
flash_attn_varlen_func(..., out=out)
# after
out = out.contiguous()
flash_attn_varlen_func(..., out=out)

if out is not None and out.stride(-1) != 1:\n    out = out.contiguous()

def valid_out(out) -> bool:\n    return out is None or (not out.requires_grad and out.stride(-1) == 1)

When it happens

Trigger: Passing out=t where t.stride(-1) != 1, e.g. a transposed or sliced tensor like out=some[:, :, 0:head_dim:2] or out of a permuted layout.

Common situations: Reusing a transposed activation buffer as out; column-major slices from prior ops.

Related errors

• out must not require gradients
• out must have stride 1 in the last dimension
• v_cache must be provided
• q can only be None when only_qv=True
• q must be provided unless qv is provided with only_qv=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bbee3ff34cc3a47a. Report an issue: GitHub.