{"record":{"id":"760f4194daa96b62","repo":"sgl-project/sglang","slug":"mlx-async-runner-does-not-support-forward-mode-f","errorCode":null,"errorMessage":"MLX async runner does not support forward mode: {forward_mode}","messagePattern":"MLX async runner does not support forward mode: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/hardware_backend/mlx/tp_worker.py","lineNumber":452,"sourceCode":"                pending_decode.lazy_tokens,\n                *lazy_logprob_arrays(pending_decode.lazy_logprobs),\n            )\n            return MlxLaunch(\n                lazy_tokens=pending_decode.lazy_tokens,\n                prefills=[],\n                extends=[],\n                decode=pending_decode,\n                mode=\"decode\",\n            )\n\n        if forward_mode.is_extend():\n            # TODO (changminbark): Implement per-batch flushing using prefix_slot_ids\n            # Ensure the pool is up-to-date before pool-backed attention\n            # reads it for prefix-cached prefills. Mirror the sync path.\n            self._mlx_runner.flush_all_decode_kv()\n            return self._async_extend_batch(batch)\n\n        raise ValueError(\n            f\"MLX async runner does not support forward mode: {forward_mode}\"\n        )\n\n    def _async_extend_batch(self, batch: ScheduleBatch) -> MlxLaunch:\n        \"\"\"Launch each request in an EXTEND batch lazily and kick GPU work.\"\"\"\n        reqs = batch.reqs\n        input_ids_cpu = batch.input_ids.cpu().tolist()\n        out_cache_loc_cpu = batch.out_cache_loc.cpu().tolist()\n        extend_seq_lens = batch.extend_lens\n        edit_rows = self._build_logit_edit_rows(batch)\n        logprob_rows = self._logprob_rows(batch)\n\n        offset = 0\n        slot_offset = 0\n        pending_prefills: list[MlxPendingPrefill] = []\n        pending_extends: list[MlxPendingExtend] = []\n        mixed_decode_rids: list[str] = []\n        # Genuine decode steps mixed into this extend batch; see","sourceCodeStart":434,"sourceCodeEnd":470,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/hardware_backend/mlx/tp_worker.py#L434-L470","documentation":"The MLX async forward dispatcher in tp_worker handles only a fixed set of forward modes (decode/extend paths that end in _async_extend_batch or flush+extend). Any other ForwardMode reaching async_forward_batch_generation_mlx raises this ValueError, guarding against unimplemented async paths.","triggerScenarios":"Calling the MLX async runner (via _launch_fresh/_forward_batch_generation_mlx) with a ForwardMode other than the supported decode/extend ones — e.g. idle mode, mixed chunk batches routed to the async path, or a new ForwardMode enum value added without an MLX branch.","commonSituations":"Enabling speculative decoding, mixed-chunk, or other scheduler features that emit exotic forward modes while SGLANG_USE_MLX is on; upgrading SGLang where a new ForwardMode was introduced but MLX support lags.","solutions":["Disable features that produce unsupported forward modes (e.g. --disable-mixed-chunk, turn off speculative decoding / idle mode) when using the MLX backend","Check server args for async/overlap scheduler interactions and fall back to the synchronous path","Report/file an issue with the ForwardMode name printed in the message so the branch can be added","As a stopgap, unset SGLANG_USE_MLX and run a supported backend"],"exampleFix":"# before\npython -m sglang.launch_server ... --enable-mixed-chunk  # mixed mode hits MLX async runner\n# after\npython -m sglang.launch_server ...  # mixed-chunk disabled for MLX backend","handlingStrategy":"fallback","validationCode":"SUPPORTED_MLX_MODES = {ForwardMode.DECODE, ForwardMode.EXTEND}\nassert batch.forward_mode.is_decode or batch.forward_mode.is_extend, (\n    f\"MLX async runner cannot handle {batch.forward_mode}\")","typeGuard":null,"tryCatchPattern":"try:\n    launch = self.async_forward_batch_generation_mlx(batch)\nexcept ValueError as e:\n    if \"forward mode\" in str(e):\n        launch = self._forward_batch_generation_mlx(batch)  # sync fallback\n    else:\n        raise","preventionTips":["Don't combine SGLANG_USE_MLX with mixed-chunk/speculative/idle modes until supported","Log forward_mode when integrating new scheduler features with MLX","Watch MLX backend release notes when new ForwardModes land"],"tags":["mlx","forward-mode","async-scheduler","sglang","unsupported-feature"],"backgroundTag":"unsupported-enum-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}