{"record":{"id":"6afd965ffbb35afd","repo":"sgl-project/sglang","slug":"server-failed-to-start-within-the-timeout-period","errorCode":null,"errorMessage":"Server failed to start within the timeout period.","messagePattern":"Server failed to start within the timeout period\\.","errorType":"exception","errorClass":"TimeoutError","httpStatus":null,"severity":"error","filePath":"python/sglang/lang/backend/runtime_endpoint.py","lineNumber":438,"sourceCode":"        with requests.Session() as session:\n            while time.time() - start_time < launch_timeout:\n                try:\n                    response = session.get(f\"{self.url}/health_generate\")\n                    if response.status_code == 200:\n                        break\n                except requests.RequestException:\n                    pass\n\n                if not proc.is_alive():\n                    self.shutdown()\n                    raise RuntimeError(\n                        \"Initialization failed. Please see the error messages above.\"\n                    )\n\n                time.sleep(2)\n            else:\n                self.shutdown()\n                raise TimeoutError(\"Server failed to start within the timeout period.\")\n\n        self.endpoint = RuntimeEndpoint(self.url)\n\n    def shutdown(self):\n        from sglang.srt.utils import kill_process_tree\n\n        if self.pid is not None:\n            # Note(kpham-sgl): __del__ routes here, so the reap wait has to stay\n            # off -- blocking inside GC stalls whichever thread is allocating.\n            kill_process_tree(self.pid, wait_timeout=None)\n            self.pid = None\n\n    def start_profile(self):\n        self.endpoint.start_profile()\n\n    def stop_profile(self):\n        self.endpoint.stop_profile()\n","sourceCodeStart":420,"sourceCodeEnd":456,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/lang/backend/runtime_endpoint.py#L420-L456","documentation":"In spawn mode, RuntimeEndpoint waits in a polling loop for the child server to become healthy; if the loop's for-else exhausts (server never became ready within the timeout) it calls shutdown() and raises TimeoutError. The server process may still be alive — e.g. downloading weights slowly — just not ready in time.","triggerScenarios":"First launch of a large model whose weights must be downloaded (slow network), heavily loaded GPU host, or a server stuck in initialization; the fixed polling budget expires before health check succeeds.","commonSituations":"Cold-start downloads of multi-GB safetensors; shared GPUs where loading takes minutes; slow filesystems; oversized models.","solutions":["Pre-download the weights (huggingface-cli download <repo>) so server startup is fast.","Point at an already-running server instead of spawning (RuntimeEndpoint(\"http://host:port\")) — no timeout applies.","If spawning is required, retry after the host is less loaded / with a smaller or quantized model; check nvidia-smi and the child logs for what was slow."],"exampleFix":"# before\nbackend = sgl.RuntimeEndpoint(\"local\", model_path=\"Qwen/Qwen2.5-72B\")  # download exceeds timeout\n\n# after\n# shell: huggingface-cli download Qwen/Qwen2.5-72B\nbackend = sgl.RuntimeEndpoint(\"local\", model_path=\"Qwen/Qwen2.5-72B\")\n# or connect to a pre-started server:\nbackend = sgl.RuntimeEndpoint(\"http://localhost:30000\")","handlingStrategy":"retry","validationCode":"# avoid spawn timeout entirely: connect to pre-started server\nbackend = sgl.RuntimeEndpoint(\"http://localhost:30000\")\n# or pre-download: huggingface-cli download <model_repo>","typeGuard":null,"tryCatchPattern":"for attempt in range(3):\n    try:\n        backend = sgl.RuntimeEndpoint(\"local\", model_path=m)\n        break\n    except TimeoutError:\n        if attempt == 2: raise\n        time.sleep(30)  # weights may still be caching; retry","preventionTips":["Pre-download model weights to local cache before first launch.","Use a persistent launched server instead of spawn for large models.","Monitor the child log to see which startup phase is slow."],"tags":["frontend","timeout","server-startup","spawn","sglang"],"backgroundTag":"server-startup-timeout","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}