{"record":{"id":"823f568468c1aa88","repo":"sgl-project/sglang","slug":"backward-pass-is-not-implemented-yet-and-we-do-not","errorCode":null,"errorMessage":"Backward pass is not implemented yet and we do not have plans to implement it because we haven't figured out how to compute dg without materializing the full hidden states for all time steps.","messagePattern":"Backward pass is not implemented yet and we do not have plans to implement it because we haven't figured out how to compute dg without materializing the full hidden states for all time steps\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent.py","lineNumber":746,"sourceCode":"        o, final_state = fused_recurrent_gated_delta_rule_fwd(\n            q=q,\n            k=k,\n            v=v,\n            g=g,\n            beta=beta,\n            scale=scale,\n            initial_state=initial_state,\n            output_final_state=output_final_state,\n            use_qk_l2norm_in_kernel=use_qk_l2norm_in_kernel,\n            cu_seqlens=cu_seqlens,\n        )\n\n        return o, final_state\n\n    @staticmethod\n    @input_guard\n    def backward(ctx, do, dht):\n        raise NotImplementedError(\n            \"Backward pass is not implemented yet and we do not have plans to implement it \"\n            \"because we haven't figured out how to compute dg without materializing the full \"\n            \"hidden states for all time steps.\"\n        )\n\n\ndef fused_recurrent_gated_delta_rule(\n    q: torch.Tensor,\n    k: torch.Tensor,\n    v: torch.Tensor,\n    g: torch.Tensor,\n    beta: torch.Tensor = None,\n    scale: float = None,\n    initial_state: torch.Tensor = None,\n    output_final_state: bool = False,\n    cu_seqlens: Optional[torch.LongTensor] = None,\n    use_qk_l2norm_in_kernel: bool = False,\n) -> Tuple[torch.Tensor, torch.Tensor]:","sourceCodeStart":728,"sourceCodeEnd":764,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent.py#L728-L764","documentation":"The autograd Function wrapping fused_recurrent (gated delta rule / KDA) explicitly does not implement backward: computing gradients w.r.t. gates would require materializing hidden states for all timesteps, which is memory-prohibitive. Calling .backward() or loss.backward() on outputs of this op raises NotImplementedError.","triggerScenarios":"Using fused_recurrent outputs in a loss and calling backward() during training; running an autograd-enabled graph that reaches this op's backward node.","commonSituations":"Trying to fine-tune/train a linear-attention model through the inference-optimized recurrent kernel; tests that accidentally build a grad graph; forgetting torch.no_grad() in benchmark/profiling code.","solutions":["Wrap inference calls in torch.no_grad() / torch.inference_mode() so backward is never invoked","For training, use a differentiable reference implementation or a chunked kernel that supports backward","Detach outputs if downstream code computes losses you don't actually need to backprop"],"exampleFix":"// before\no, s = fused_recurrent_gated_delta_rule(q, k, v, ...)\nloss = o.sum(); loss.backward()  # NotImplementedError\n// after\nwith torch.no_grad():\n    o, s = fused_recurrent_gated_delta_rule(q, k, v, ...)\n# inference only; use a differentiable path for training","handlingStrategy":"validation","validationCode":"assert not torch.is_grad_enabled(), 'kernel is inference-only'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Wrap all fused_recurrent calls in torch.inference_mode()","Detach outputs before returning from inference modules"],"tags":["pytorch","autograd","backward","linear-attention","not-implemented"],"backgroundTag":"autograd-backward-not-implemented","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}