sgl-project/sglang · error · TimeoutError

Server failed to start within the timeout period.

Error message

Server failed to start within the timeout period.

What it means

In spawn mode, RuntimeEndpoint waits in a polling loop for the child server to become healthy; if the loop's for-else exhausts (server never became ready within the timeout) it calls shutdown() and raises TimeoutError. The server process may still be alive — e.g. downloading weights slowly — just not ready in time.

Source

Thrown at python/sglang/lang/backend/runtime_endpoint.py:438

        with requests.Session() as session:
            while time.time() - start_time < launch_timeout:
                try:
                    response = session.get(f"{self.url}/health_generate")
                    if response.status_code == 200:
                        break
                except requests.RequestException:
                    pass

                if not proc.is_alive():
                    self.shutdown()
                    raise RuntimeError(
                        "Initialization failed. Please see the error messages above."
                    )

                time.sleep(2)
            else:
                self.shutdown()
                raise TimeoutError("Server failed to start within the timeout period.")

        self.endpoint = RuntimeEndpoint(self.url)

    def shutdown(self):
        from sglang.srt.utils import kill_process_tree

        if self.pid is not None:
            # Note(kpham-sgl): __del__ routes here, so the reap wait has to stay
            # off -- blocking inside GC stalls whichever thread is allocating.
            kill_process_tree(self.pid, wait_timeout=None)
            self.pid = None

    def start_profile(self):
        self.endpoint.start_profile()

    def stop_profile(self):
        self.endpoint.stop_profile()

View on GitHub (pinned to 0132848349)

Solutions

1. Pre-download the weights (huggingface-cli download <repo>) so server startup is fast.
2. Point at an already-running server instead of spawning (RuntimeEndpoint("http://host:port")) — no timeout applies.
3. If spawning is required, retry after the host is less loaded / with a smaller or quantized model; check nvidia-smi and the child logs for what was slow.

Example fix

# before
backend = sgl.RuntimeEndpoint("local", model_path="Qwen/Qwen2.5-72B")  # download exceeds timeout

# after
# shell: huggingface-cli download Qwen/Qwen2.5-72B
backend = sgl.RuntimeEndpoint("local", model_path="Qwen/Qwen2.5-72B")
# or connect to a pre-started server:
backend = sgl.RuntimeEndpoint("http://localhost:30000")

# avoid spawn timeout entirely: connect to pre-started server
backend = sgl.RuntimeEndpoint("http://localhost:30000")
# or pre-download: huggingface-cli download <model_repo>

for attempt in range(3):
    try:
        backend = sgl.RuntimeEndpoint("local", model_path=m)
        break
    except TimeoutError:
        if attempt == 2: raise
        time.sleep(30)  # weights may still be caching; retry

When it happens

Trigger: First launch of a large model whose weights must be downloaded (slow network), heavily loaded GPU host, or a server stuck in initialization; the fixed polling budget expires before health check succeeds.

Common situations: Cold-start downloads of multi-GB safetensors; shared GPUs where loading takes minutes; slow filesystems; oversized models.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Initialization failed. Please see the error messages above.
• DeepGEMM Kernels compilation timeout.\n\nFeel free and pleas
• Waiting for main node timeout!
• Crusoe API key required. Pass api_key= or set CRUSOE_API_KEY
• This use case is not supported if api speculative execution

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6afd965ffbb35afd. Report an issue: GitHub.