sgl-project/sglang · error · NotImplementedError

Mamba2AttnBackend's forward is called directly instead of th

Error message

Mamba2AttnBackend's forward is called directly instead of through HybridLinearAttnBackend, as it supports mixed prefill and decode

What it means

Mamba2AttnBackend implements a single unified forward() that natively handles mixed prefill+decode batches (SSM state kernels process both), so the AttentionBackend interface's forward_decode/forward_extend split does not apply. Calling forward_decode directly raises NotImplementedError to enforce that dispatch go through HybridLinearAttnBackend, which calls forward() once.

Source

Thrown at python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py:951

                    self.forward_metadata,
                )

            if self.forward_metadata.num_decodes > 0:
                num_decodes = self.forward_metadata.num_decodes
                track_mamba_states_if_needed(
                    layer_cache.conv[0],
                    layer_cache.temporal,
                    self.forward_metadata.mamba_cache_indices[-num_decodes:],
                    forward_batch.mamba_track_mask[-num_decodes:],
                    self.forward_metadata.mamba_track_indices[-num_decodes:],
                    num_decodes,
                    check_freed_slots=self.enable_unified_memory,
                )

        return mixer_out

    def forward_decode(self, *args, **kwargs):
        raise NotImplementedError(
            "Mamba2AttnBackend's forward is called directly instead of through HybridLinearAttnBackend, as it supports mixed prefill and decode"
        )

    def forward_extend(self, *args, **kwargs):
        raise NotImplementedError(
            "Mamba2AttnBackend's forward is called directly instead of through HybridLinearAttnBackend, as it supports mixed prefill and decode"
        )


class HybridLinearAttnBackend(AttentionBackend):
    """Manages a full and linear attention backend"""

    def __init__(
        self,
        full_attn_backend: AttentionBackend,
        linear_attn_backend: MambaAttnBackendBase,
        full_attn_layers: list[int],
    ):

View on GitHub (pinned to 0132848349)

Solutions

1. Route calls through HybridLinearAttnBackend.forward / the mamba backend's forward() which handles mixed prefill+decode
2. If writing generic dispatch code, check isinstance/backend capability and call forward() for mamba backends
3. Remove any manual if-mode-then-forward_decode branching for Mamba2AttnBackend

Example fix

# before
if forward_batch.forward_mode.is_decode():
    out = mamba2_backend.forward_decode(...)
# after
out = mamba2_backend.forward(...)  # handles mixed prefill + decode

from sglang.srt.layers.attention.hybrid_linear_attn_backend import Mamba2AttnBackend, HybridLinearAttnBackend

if isinstance(backend, Mamba2AttnBackend):
    out = backend.forward(...)
elif isinstance(backend, HybridLinearAttnBackend):
    out = backend.forward(...)
else:
    out = backend.forward_decode(...) if mode.is_decode() else backend.forward_extend(...)

def has_unified_forward(backend) -> bool:
    return isinstance(backend, (Mamba2AttnBackend, HybridLinearAttnBackend))

When it happens

Trigger: Calling Mamba2AttnBackend.forward_decode(...) directly (e.g. custom scheduler code or a generic runner that dispatches by forward mode), instead of invoking its forward() via HybridLinearAttnBackend.

Common situations: Writing a custom attention runner or porting code that assumes the forward_extend/forward_decode contract; refactoring that bypasses HybridLinearAttnBackend and talks to the mamba sub-backend directly.

Related errors

• Invalid forward mode: {forward_batch.forward_mode=}
• quantize_and_serve functionality is currently disabled due t
• Short-conv hybrid models (ZAYA1 CCA, LFM2 / LFM2-MoE) are no
• DSA indexer only supports CUDA, HIP, and NPU
• Invalid forward mode: {forward_mode=}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2e6aa6d985131a49. Report an issue: GitHub.