{"record":{"id":"bbee3ff34cc3a47a","repo":"sgl-project/sglang","slug":"out-must-have-stride-1-in-the-last-dimension","errorCode":null,"errorMessage":"out must have stride 1 in the last dimension","messagePattern":"out must have stride 1 in the last dimension","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attention_v4_sm120.py","lineNumber":73,"sourceCode":"        resolve_runtime_policy(\n            device_capability=device_capability,\n            deterministic=deterministic,\n        )\n    )\n    return FlashAttentionV4SM120RuntimePolicy(\n        num_splits=num_splits,\n        decode_num_splits=decode_num_splits,\n        decode_uses_static_max_seqlen_k=decode_uses_static_max_seqlen_k,\n    )\n\n\ndef _validate_out_contract(out: Optional[torch.Tensor]) -> None:\n    if out is None:\n        return\n    if out.requires_grad:\n        raise ValueError(\"out must not require gradients\")\n    if out.stride(-1) != 1:\n        raise ValueError(\"out must have stride 1 in the last dimension\")\n\n\n@debug_kernel_api\ndef flash_attn_varlen_func(\n    q: torch.Tensor,\n    k: torch.Tensor,\n    v: torch.Tensor,\n    cu_seqlens_q: Optional[torch.Tensor] = None,\n    cu_seqlens_k: Optional[torch.Tensor] = None,\n    qv: Optional[torch.Tensor] = None,\n    seqused_q: Optional[torch.Tensor] = None,\n    seqused_k: Optional[torch.Tensor] = None,\n    max_seqlen_q: Optional[int] = None,\n    max_seqlen_k: Optional[int] = None,\n    page_table: Optional[torch.Tensor] = None,\n    softmax_scale: Optional[float] = None,\n    causal: bool = False,\n    softcap: Optional[float] = None,","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attention_v4_sm120.py#L55-L91","documentation":"FA4 sm120 kernels require the out tensor's last dimension to be contiguous (stride 1) for coalesced vectorized stores; any other last-dim stride raises ValueError in _validate_out_contract.","triggerScenarios":"Passing out=t where t.stride(-1) != 1, e.g. a transposed or sliced tensor like out=some[:, :, 0:head_dim:2] or out of a permuted layout.","commonSituations":"Reusing a transposed activation buffer as out; column-major slices from prior ops.","solutions":["Make out contiguous: out = out.contiguous()","Allocate a fresh contiguous buffer: out = torch.empty_like(q)","Restructure so the last dim is unit-stride before calling"],"exampleFix":"# before\nout = buf.transpose(1, 2)  # last-dim stride != 1\nflash_attn_varlen_func(..., out=out)\n# after\nout = out.contiguous()\nflash_attn_varlen_func(..., out=out)","handlingStrategy":"validation","validationCode":"if out is not None and out.stride(-1) != 1:\\n    out = out.contiguous()","typeGuard":"def valid_out(out) -> bool:\\n    return out is None or (not out.requires_grad and out.stride(-1) == 1)","tryCatchPattern":null,"preventionTips":["Never pass transposed/sliced views as out","Allocate fresh contiguous buffers for kernel outputs"],"tags":["flash-attention","fa4","stride","memory-layout","validation"],"backgroundTag":"output-tensor-contract-violation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}