{"record":{"id":"d441c32a6beff2fd","repo":"sgl-project/sglang","slug":"out-must-not-require-gradients-d441c3","errorCode":null,"errorMessage":"out must not require gradients","messagePattern":"out must not require gradients","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/interface.py","lineNumber":644,"sourceCode":"    else:\n        # num_head contiguous better for MQA in MLA absorbed\n        lse_shape = (\n            (batch_size, seqlen_q, num_head)\n            if cu_seqlens_q is None\n            else (total_q, num_head)\n        )\n\n    if out is None:\n        out = torch.empty(\n            *q_batch_seqlen_shape,\n            num_head,\n            head_dim_v,\n            dtype=out_torch_dtype,\n            device=device,\n        )\n    else:\n        if out.requires_grad:\n            raise ValueError(\"out must not require gradients\")\n        if out.stride(-1) != 1:\n            raise ValueError(\"out must have stride 1 in the last dimension\")\n        _validate_tensor(\n            out,\n            \"out\",\n            (*q_batch_seqlen_shape, num_head, head_dim_v),\n            out_torch_dtype,\n            device,\n        )\n\n    if lse is None:\n        lse = (\n            torch.empty(lse_shape, dtype=torch.float32, device=device)\n            if requires_grad or return_lse\n            else None\n        )\n    elif lse is not None:\n        _validate_tensor(lse, \"lse\", lse_shape, torch.float32, device)","sourceCodeStart":626,"sourceCodeEnd":662,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/interface.py#L626-L662","documentation":"When the caller supplies a preallocated out tensor, the FA kernel writes in-place and cannot propagate gradients through it — autograd would silently be wrong. Hence out.requires_grad must be False.","triggerScenarios":"Passing out= tensor that was created with requires_grad=True (directly or via a differentiable factory like torch.zeros(..., requires_grad=True)).","commonSituations":"Training loops where all buffers are allocated with requires_grad; reusing a leaf parameter as the output buffer.","solutions":["Allocate out with requires_grad=False (default)","Call out.detach() (or out.requires_grad_(False)) before passing","Omit out and let the kernel allocate, so autograd flows normally"],"exampleFix":"// before\nout = torch.empty(shape, requires_grad=True)\nfa(..., out=out)\n// after\nout = torch.empty(shape)\nfa(..., out=out)","handlingStrategy":"validation","validationCode":"if out is not None and out.requires_grad:\n    out = out.detach()","typeGuard":"def out_tensor_safe(out) -> bool: return out is None or not out.requires_grad","tryCatchPattern":null,"preventionTips":["Never allocate buffers with requires_grad=True unless they're parameters","Detach any reused buffer before passing as out="],"tags":["flash-attention","autograd","out-tensor","validation"],"backgroundTag":"invalid-argument-state","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}