{"record":{"id":"1a6dd0b0cc7b7b5d","repo":"sgl-project/sglang","slug":"out-must-not-require-gradients","errorCode":null,"errorMessage":"out must not require gradients","messagePattern":"out must not require gradients","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attention_v4_sm120.py","lineNumber":71,"sourceCode":"        )\n    num_splits, decode_num_splits, decode_uses_static_max_seqlen_k = (\n        resolve_runtime_policy(\n            device_capability=device_capability,\n            deterministic=deterministic,\n        )\n    )\n    return FlashAttentionV4SM120RuntimePolicy(\n        num_splits=num_splits,\n        decode_num_splits=decode_num_splits,\n        decode_uses_static_max_seqlen_k=decode_uses_static_max_seqlen_k,\n    )\n\n\ndef _validate_out_contract(out: Optional[torch.Tensor]) -> None:\n    if out is None:\n        return\n    if out.requires_grad:\n        raise ValueError(\"out must not require gradients\")\n    if out.stride(-1) != 1:\n        raise ValueError(\"out must have stride 1 in the last dimension\")\n\n\n@debug_kernel_api\ndef flash_attn_varlen_func(\n    q: torch.Tensor,\n    k: torch.Tensor,\n    v: torch.Tensor,\n    cu_seqlens_q: Optional[torch.Tensor] = None,\n    cu_seqlens_k: Optional[torch.Tensor] = None,\n    qv: Optional[torch.Tensor] = None,\n    seqused_q: Optional[torch.Tensor] = None,\n    seqused_k: Optional[torch.Tensor] = None,\n    max_seqlen_q: Optional[int] = None,\n    max_seqlen_k: Optional[int] = None,\n    page_table: Optional[torch.Tensor] = None,\n    softmax_scale: Optional[float] = None,","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attention_v4_sm120.py#L53-L89","documentation":"The FA4 sm120 wrapper can write into a caller-provided out tensor, but requires it to be a plain non-autograd buffer: out.requires_grad must be False. Otherwise it raises ValueError to avoid autograd silent-breakage.","triggerScenarios":"Calling flash_attn_varlen_func/flash_attn_with_kvcache(out=t) where t.requires_grad is True (e.g. a leaf tensor with grad enabled).","commonSituations":"Passing a parameters-derived or grad-tracking buffer as out during training/debugging; running under autograd-enabled contexts without torch.no_grad().","solutions":["Detach the buffer: out = out.detach() before passing","Allocate out with torch.empty(..., requires_grad=False)","Wrap the call in torch.no_grad() so buffers don't track grad"],"exampleFix":"# before\nout = q.new_empty(...); out.requires_grad_(True)\nflash_attn_varlen_func(..., out=out)\n# after\nout = out.detach()\nflash_attn_varlen_func(..., out=out)","handlingStrategy":"validation","validationCode":"if out is not None and out.requires_grad:\\n    out = out.detach()","typeGuard":"def valid_out(out) -> bool:\\n    return out is None or (not out.requires_grad and out.stride(-1) == 1)","tryCatchPattern":null,"preventionTips":["Allocate out with torch.empty under no_grad","Detach reused buffers before passing as out"],"tags":["flash-attention","fa4","autograd","out-tensor","validation"],"backgroundTag":"output-tensor-contract-violation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}