{"record":{"id":"2e6aa6d985131a49","repo":"sgl-project/sglang","slug":"mamba2attnbackend-s-forward-is-called-directly-ins","errorCode":null,"errorMessage":"Mamba2AttnBackend's forward is called directly instead of through HybridLinearAttnBackend, as it supports mixed prefill and decode","messagePattern":"Mamba2AttnBackend's forward is called directly instead of through HybridLinearAttnBackend, as it supports mixed prefill and decode","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py","lineNumber":951,"sourceCode":"                    self.forward_metadata,\n                )\n\n            if self.forward_metadata.num_decodes > 0:\n                num_decodes = self.forward_metadata.num_decodes\n                track_mamba_states_if_needed(\n                    layer_cache.conv[0],\n                    layer_cache.temporal,\n                    self.forward_metadata.mamba_cache_indices[-num_decodes:],\n                    forward_batch.mamba_track_mask[-num_decodes:],\n                    self.forward_metadata.mamba_track_indices[-num_decodes:],\n                    num_decodes,\n                    check_freed_slots=self.enable_unified_memory,\n                )\n\n        return mixer_out\n\n    def forward_decode(self, *args, **kwargs):\n        raise NotImplementedError(\n            \"Mamba2AttnBackend's forward is called directly instead of through HybridLinearAttnBackend, as it supports mixed prefill and decode\"\n        )\n\n    def forward_extend(self, *args, **kwargs):\n        raise NotImplementedError(\n            \"Mamba2AttnBackend's forward is called directly instead of through HybridLinearAttnBackend, as it supports mixed prefill and decode\"\n        )\n\n\nclass HybridLinearAttnBackend(AttentionBackend):\n    \"\"\"Manages a full and linear attention backend\"\"\"\n\n    def __init__(\n        self,\n        full_attn_backend: AttentionBackend,\n        linear_attn_backend: MambaAttnBackendBase,\n        full_attn_layers: list[int],\n    ):","sourceCodeStart":933,"sourceCodeEnd":969,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py#L933-L969","documentation":"Mamba2AttnBackend implements a single unified forward() that natively handles mixed prefill+decode batches (SSM state kernels process both), so the AttentionBackend interface's forward_decode/forward_extend split does not apply. Calling forward_decode directly raises NotImplementedError to enforce that dispatch go through HybridLinearAttnBackend, which calls forward() once.","triggerScenarios":"Calling Mamba2AttnBackend.forward_decode(...) directly (e.g. custom scheduler code or a generic runner that dispatches by forward mode), instead of invoking its forward() via HybridLinearAttnBackend.","commonSituations":"Writing a custom attention runner or porting code that assumes the forward_extend/forward_decode contract; refactoring that bypasses HybridLinearAttnBackend and talks to the mamba sub-backend directly.","solutions":["Route calls through HybridLinearAttnBackend.forward / the mamba backend's forward() which handles mixed prefill+decode","If writing generic dispatch code, check isinstance/backend capability and call forward() for mamba backends","Remove any manual if-mode-then-forward_decode branching for Mamba2AttnBackend"],"exampleFix":"# before\nif forward_batch.forward_mode.is_decode():\n    out = mamba2_backend.forward_decode(...)\n# after\nout = mamba2_backend.forward(...)  # handles mixed prefill + decode","handlingStrategy":"type-guard","validationCode":"from sglang.srt.layers.attention.hybrid_linear_attn_backend import Mamba2AttnBackend, HybridLinearAttnBackend\n\nif isinstance(backend, Mamba2AttnBackend):\n    out = backend.forward(...)\nelif isinstance(backend, HybridLinearAttnBackend):\n    out = backend.forward(...)\nelse:\n    out = backend.forward_decode(...) if mode.is_decode() else backend.forward_extend(...)","typeGuard":"def has_unified_forward(backend) -> bool:\n    return isinstance(backend, (Mamba2AttnBackend, HybridLinearAttnBackend))","tryCatchPattern":null,"preventionTips":["Never call forward_decode/forward_extend on mamba sub-backends","Centralize attention dispatch in one helper that knows backend capabilities","Add isinstance guards in custom runners that bypass HybridLinearAttnBackend"],"tags":["mamba","hybrid-linear-attention","interface-contract","not-implemented","sglang"],"backgroundTag":"wrong-backend-dispatch-call","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}