{"record":{"id":"b24e5af7781b6bd2","repo":"sgl-project/sglang","slug":"invalid-forward-mode-forward-mode-b24e5a","errorCode":null,"errorMessage":"Invalid forward mode: {forward_mode=}","messagePattern":"Invalid forward mode: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py","lineNumber":524,"sourceCode":"        spec_info: Optional[Union[EagleDraftInput, EagleVerifyInput]],\n    ):\n        if forward_mode.is_decode_or_idle():\n            self.query_start_loc_list[bs - 1].copy_(\n                self.cached_cuda_graph_decode_query_start_loc[: bs + 1]\n            )\n        elif forward_mode.is_target_verify():\n            ragged_layout = (\n                spec_info.ragged_verify_layout if spec_info is not None else None\n            )\n            if ragged_layout is not None:\n                # Ragged capture: qsl from the runner's synthetic layout.\n                self.query_start_loc_list[bs - 1].copy_(ragged_layout.qo_indptr_device)\n            else:\n                self.query_start_loc_list[bs - 1].copy_(\n                    self.cached_cuda_graph_verify_query_start_loc[: bs + 1]\n                )\n        else:\n            raise ValueError(f\"Invalid forward mode: {forward_mode=}\")\n        mamba_indices = self.req_to_token_pool.get_mamba_indices(req_pool_indices)\n        # Captured Mamba kernels read state_indices_list as PHYSICAL ids; translate\n        # before copying (no-op for non-unified pool).\n        mamba_indices = self._translate_mamba_indices(mamba_indices)\n        self.state_indices_list[bs - 1][: len(mamba_indices)].copy_(mamba_indices)\n\n        # Capture records the pointer to the static per-bs buffers; their zeros are\n        # overwritten in-place by _replay_metadata before each replay. None when off.\n        replayssm_write_pos = (\n            self.replayssm_write_pos_list[bs - 1]\n            if self.replayssm_write_pos_list is not None\n            else None\n        )\n        replayssm_force_flush = (\n            self.replayssm_force_flush_list[bs - 1]\n            if self.replayssm_force_flush_list is not None\n            else None\n        )","sourceCodeStart":506,"sourceCodeEnd":542,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/hybrid_linear_attn_backend.py#L506-L542","documentation":"During CUDA graph capture, HybridLinearAttnBackend._capture_metadata pre-populates static buffers (query_start_loc, state_indices) per batch size for each supported forward mode. An unrecognized forward_mode at capture time raises this ValueError, preventing a graph from being captured with metadata layouts the backend cannot reproduce at replay time.","triggerScenarios":"init_forward_metadata_capture_cpu_graph is invoked with a forward_mode outside the supported set (e.g. a new speculative mode enabled with --cuda-graph on a hybrid/mamba model), hitting the final else branch.","commonSituations":"Enabling cuda graph with a hybrid linear attention model plus a new or experimental decoding mode that the backend's capture path was not updated for; sglang version skew between core modes and the mamba backend.","solutions":["Disable CUDA graph (--disable-cuda-graph) for the unsupported mode on hybrid models","Update _capture_metadata's mode chain to handle the new forward_mode (mirror the replay-side layout)","Use a sglang version where this decoding mode and the hybrid backend are compatible"],"exampleFix":"# before\npython -m sglang.launch_server --model hybridenabled --speculative-algorithm NEW --cuda-graph-max-bs ...\n# after\npython -m sglang.launch_server --model hybridenabled --speculative-algorithm NEW --disable-cuda-graph","handlingStrategy":"validation","validationCode":"# before launching with cuda graphs\nassert backend_handles_mode(mode), f'cannot capture graph for {mode}'\n# or just: server_args.disable_cuda_graph = True when using unsupported mode","typeGuard":null,"tryCatchPattern":"try:\n    backend.init_forward_metadata_capture_cpu_graph(...)\nexcept ValueError as e:\n    if 'Invalid forward mode' in str(e):\n        disable_cuda_graphs_and_rerun()\n    else:\n        raise","preventionTips":["Disable CUDA graphs when trying new speculative modes on hybrid models","Keep capture/replay mode chains in sync when editing the backend","Pin to stable sglang releases for hybrid + speculative combos"],"tags":["cuda-graph","forward-mode","hybrid-linear-attention","capture","sglang"],"backgroundTag":"unhandled-enum-value-dispatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}