{"record":{"id":"a9a5663f14c78791","repo":"hiyouga/LlamaFactory","slug":"sglang-server-initialization-failed-str-e","errorCode":null,"errorMessage":"SGLang server initialization failed: {str(e)}.","messagePattern":"SGLang server initialization failed: (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/chat/sglang_engine.py","lineNumber":128,"sourceCode":"            self.server_process, port = launch_server_cmd(launch_cmd)\n            self.base_url = f\"http://localhost:{port}\"\n            atexit.register(self._cleanup_server)\n\n            logger.info_rank0(f\"Waiting for SGLang server to be ready at {self.base_url}\")\n            wait_for_server(self.base_url, timeout=300)\n            logger.info_rank0(f\"SGLang server initialized successfully at {self.base_url}\")\n            try:\n                response = requests.get(f\"{self.base_url}/get_model_info\", timeout=5)\n                if response.status_code == 200:\n                    model_info = response.json()\n                    logger.info(f\"SGLang server model info: {model_info}\")\n            except Exception as e:\n                logger.debug(f\"Note: could not get model info: {str(e)}\")\n\n        except Exception as e:\n            logger.error(f\"Failed to start SGLang server: {str(e)}\")\n            self._cleanup_server()  # make sure to clean up any started process\n            raise RuntimeError(f\"SGLang server initialization failed: {str(e)}.\")\n\n    def _cleanup_server(self):\n        r\"\"\"Clean up the server process when the engine is destroyed.\"\"\"\n        if hasattr(self, \"server_process\") and self.server_process:\n            try:\n                logger.info(\"Terminating SGLang server process\")\n                terminate_process(self.server_process)\n                logger.info(\"SGLang server process terminated\")\n            except Exception as e:\n                logger.warning(f\"Error terminating SGLang server: {str(e)}\")\n\n    async def _generate(\n        self,\n        messages: list[dict[str, str]],\n        system: Optional[str] = None,\n        tools: Optional[str] = None,\n        images: Optional[list[\"ImageInput\"]] = None,\n        videos: Optional[list[\"VideoInput\"]] = None,","sourceCodeStart":110,"sourceCodeEnd":146,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/chat/sglang_engine.py#L110-L146","documentation":"Raised when the embedded SGLang inference server fails to start inside SGLangEngine initialization. Any exception thrown while launching the server subprocess (import failure, bad arguments, port conflict, model load error, or the server crashing/exiting before becoming healthy) is caught, the partially started process is cleaned up via _cleanup_server(), and the original exception text is rewrapped in a RuntimeError. The underlying cause is visible in the preceding 'Failed to start SGLang server' log line.","triggerScenarios":"Constructing SGLangEngine(model_args) when: sglang is not installed or version-mismatched; the model path is wrong or the model OOMs on load; the chosen port is already in use; an invalid engine argument is passed to the sglang server launch command; or the server process dies during its startup wait.","commonSituations":"Running llamafactory-cli chat/api with an sglang engine on a machine without a compatible sglang build or CUDA driver; specifying a quantization or max-model-len the GPU cannot satisfy; leftover process holding the server port; sglang/transformers version skew after upgrading LlamaFactory.","solutions":["Read the 'Failed to start SGLang server: ...' error line above the traceback — it carries the root cause from the sglang subprocess.","Verify sglang is installed and importable in the same Python env: python -c \"import sglang; print(sglang.__version__)\".","Check GPU memory and lower max_model_len / disable quantization mismatch; watch the sglang server log output for CUDA OOM.","Free the port or let the engine pick another (kill stale sglang processes: pkill -f sglang).","Align versions: reinstall sglang matching the LlamaFactory requirement for your transformers version."],"exampleFix":"# before\nengine = SGLangEngine(model_args)  # RuntimeError: SGLang server initialization failed\n\n# after\nimport subprocess\nsubprocess.run([\"pkill\", \"-f\", \"sglang\"], check=False)  # free stale port/process\nengine = SGLangEngine(model_args)","handlingStrategy":"retry","validationCode":"import importlib.util, socket\nspec = importlib.util.find_spec(\"sglang\")\nassert spec is not None, \"sglang is not installed\"\nwith socket.socket() as s:\n    s.bind((\"127.0.0.1\", 0))  # probe: machine can allocate sockets\nprint(\"sglang importable:\", spec.origin)","typeGuard":null,"tryCatchPattern":"try:\n    engine = SGLangEngine(model_args)\nexcept RuntimeError as e:\n    if \"initialization failed\" not in str(e):\n        raise\n    logger.error(\"sglang start failed: %s\", e)  # message embeds root cause\n    # stale-process cleanup then one manual retry, else fall back to hf engine","preventionTips":["Smoke-test `import sglang` in the target env before launching runs.","Run one small model through the sglang engine after every dependency upgrade.","Keep GPU memory headroom for server startup; monitor with nvidia-smi.","Kill stale sglang server processes between experiments."],"tags":["sglang","server","subprocess","inference","startup"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}