{"record":{"id":"2f61984c034089bc","repo":"sgl-project/sglang","slug":"unknown-mlx-async-mode-launch-mode","errorCode":null,"errorMessage":"Unknown MLX async mode: {launch.mode}","messagePattern":"Unknown MLX async mode: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/hardware_backend/mlx/tp_worker.py","lineNumber":658,"sourceCode":"                mixed_tokens = self._mlx_runner.decode_batch_finalize(decode)\n                decode_map = {\n                    rid: tok for rid, tok in zip(decode.req_ids, mixed_tokens)\n                }\n                self._collect_step_logprobs(\n                    step_logprob_rows, decode.lazy_logprobs, decode.req_ids\n                )\n\n            next_tokens_list = []\n            for req in reqs:\n                if req.rid in decode_map:\n                    next_tokens_list.append(decode_map[req.rid])\n                elif req.rid in extend_map:\n                    next_tokens_list.append(extend_map[req.rid])\n                else:\n                    next_tokens_list.append(prefill_map[req.rid])\n\n        else:\n            raise ValueError(f\"Unknown MLX async mode: {launch.mode}\")\n\n        next_token_ids = torch.tensor(next_tokens_list, dtype=torch.long, device=\"cpu\")\n        logits_output = (\n            self._assemble_logprob_output(step_logprob_rows, reqs)\n            if step_logprob_rows\n            else LogitsProcessorOutput(next_token_logits=None)\n        )\n        return GenerationBatchResult(\n            logits_output=logits_output,\n            next_token_ids=next_token_ids,\n            can_run_cuda_graph=False,\n        )\n","sourceCodeStart":640,"sourceCodeEnd":671,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/hardware_backend/mlx/tp_worker.py#L640-L671","documentation":"finalize_mlx_result consumes the MlxLaunch produced earlier by the async runner; it switches on launch.mode and, for any mode it does not recognize (outside decode/extend/prefill result maps), raises this ValueError. It indicates an internal contract break between the launcher and finalizer, or a new async mode added on one side only.","triggerScenarios":"An MlxLaunch reaching finalize_mlx_result with launch.mode outside the handled set — typically the same unsupported forward mode that slipped past (or was deliberately routed through) the launcher, or custom code constructing an MlxLaunch with an unregistered mode string.","commonSituations":"Extending the MLX worker with a new async mode but forgetting the finalize branch; race where a code change updated only the launch half; bugs from cherry-picks between versions.","solutions":["If hit without custom code, capture launch.mode from logs and report an SGLang issue; disable the feature triggering the exotic mode","If developing a new mode, add a matching branch in finalize_mlx_result that fills next_tokens_list/step_logprob_rows for that mode","Ensure launcher and finalizer come from the same SGLang version (no partial upgrade / mixed install)","Retry with the sync MLX path or another backend to confirm it is mode-specific"],"exampleFix":"# before: new mode launched but not finalized\nlaunch = MlxLaunch(mode=\"speculative\", ...)\nresult = worker.finalize_mlx_result(launch)  # ValueError\n# after: handle the mode\nelif launch.mode == \"speculative\":\n    next_tokens_list.append(spec_map[req.rid])","handlingStrategy":"type-guard","validationCode":"KNOWN_MODES = {\"decode\", \"extend\", \"prefill\"}  # match finalize_mlx_result branches\nassert launch.mode in KNOWN_MODES, f\"finalize cannot handle mode {launch.mode}\"","typeGuard":"def is_finalizable_mlx_launch(launch) -> bool:\n    return getattr(launch, \"mode\", None) in {\"decode\", \"extend\", \"prefill\"}","tryCatchPattern":"try:\n    out = finalize_mlx_result(launch)\nexcept ValueError as e:\n    if \"Unknown MLX async mode\" in str(e):\n        raise RuntimeError(f\"bug: launcher produced unsupported mode {launch.mode}\") from e\n    raise","preventionTips":["When adding a launch mode, add the finalize branch in the same commit","Never construct MlxLaunch objects with ad-hoc mode strings","Keep launcher/finalizer from the same SGLang version"],"tags":["mlx","async-scheduler","internal-contract","sglang"],"backgroundTag":"unhandled-dispatch-case","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}