sgl-project/sglang · error · ValueError

Invalid forward mode: {forward_batch.forward_mode=}

Error message

Invalid forward mode: {forward_batch.forward_mode=}

What it means

HybridLinearAttnBackend._forward_metadata builds per-mode metadata (SSM/mamba indices, query layouts, tracking buffers) and only handles the forward modes it knows (extend, decode, idle, and speculative modes). If forward_batch.forward_mode is anything else, it raises this ValueError to fail fast instead of building garbage metadata.

Source

Thrown at python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py:244

                )
                query_start_loc[:bs] = forward_batch.extend_start_loc
                query_start_loc[bs] = (
                    forward_batch.extend_start_loc[-1]
                    + forward_batch.extend_seq_lens[-1]
                )
                if has_mamba_track_mask:
                    track_conv_indices = self._init_track_conv_indices(
                        query_start_loc, forward_batch
                    )

                    (
                        track_ssm_h_src,
                        track_ssm_h_dst,
                        track_ssm_final_src,
                        track_ssm_final_dst,
                    ) = self._init_track_ssm_indices(mamba_cache_indices, forward_batch)
        else:
            raise ValueError(f"Invalid forward mode: {forward_batch.forward_mode=}")

        return ForwardMetadata(
            query_start_loc=query_start_loc,
            mamba_cache_indices=mamba_cache_indices,
            # Physical track destinations (None when tracking off); cuda-graph
            # supplies this via the static backend buffer in _replay_metadata.
            mamba_track_indices=getattr(forward_batch, "mamba_track_indices", None),
            retrieve_next_token=retrieve_next_token,
            retrieve_next_sibling=retrieve_next_sibling,
            retrieve_parent_token=retrieve_parent_token,
            track_conv_indices=track_conv_indices,
            track_ssm_h_src=track_ssm_h_src,
            track_ssm_h_dst=track_ssm_h_dst,
            track_ssm_final_src=track_ssm_final_src,
            track_ssm_final_dst=track_ssm_final_dst,
            has_mamba_track_mask=has_mamba_track_mask,
            replayssm_write_pos=replayssm_write_pos,
            replayssm_force_flush=replayssm_force_flush,

View on GitHub (pinned to 0132848349)

Solutions

1. Check ForwardBatch.forward_mode before dispatching; route only supported modes (EXTEND/DECODE/idle/speculative) to HybridLinearAttnBackend
2. If you added a new ForwardMode, extend the if/elif chain in _forward_metadata to handle it
3. Upgrade/downgrade sglang so backend and ForwardMode enum are in sync

SUPPORTED = {ForwardMode.EXTEND, ForwardMode.DECODE, ForwardMode.IDLE,
              ForwardMode.TARGET_VERIFY, ForwardMode.DRAFT}
assert forward_batch.forward_mode in SUPPORTED, forward_batch.forward_mode

def backend_handles_mode(mode: ForwardMode) -> bool:
    return mode in {ForwardMode.EXTEND, ForwardMode.DECODE, ForwardMode.IDLE,
                    ForwardMode.TARGET_VERIFY, ForwardMode.DRAFT}

try:
    backend.init_forward_metadata(...)
except ValueError as e:
    if 'Invalid forward mode' in str(e):
        raise UnsupportedModeForHybridBackend(forward_batch.forward_mode) from e
    raise

When it happens

Trigger: Calling init_forward_metadata on HybridLinearAttnBackend with a ForwardMode not covered by its if/elif chain — e.g. a newly added ForwardMode enum value (new speculative/draft mode) or a custom forward mode passed via a ForwardBatch.

Common situations: Adding a new forward mode to sglang (e.g. a new speculative decoding strategy or draft extension mode) without updating the hybrid linear (mamba/GDN) backend; or a version mismatch where the backend lags the core ForwardMode enum.

Related errors

• Invalid forward mode: {forward_mode=}
• Mamba2AttnBackend's forward is called directly instead of th
• MLX async runner does not support forward mode: {forward_mod
• spec_info is unset in TARGET_VERIFY mode; the extend_* metad
• Mamba storage zero-copy requires page_first layout, got {sel

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e59d2485b40387ab. Report an issue: GitHub.