sgl-project/sglang · critical · RuntimeError

Initialization failed. Please see the error messages above.

Error message

Initialization failed. Please see the error messages above.

What it means

When RuntimeEndpoint is asked to launch its own sglang server (spawn mode), it polls the child process; if the child dies during startup it shuts it down and raises RuntimeError('Initialization failed. Please see the error messages above.') — the real cause (CUDA OOM, bad model path, port conflict, missing weights) is printed by the dying server above this exception.

Source

Thrown at python/sglang/lang/backend/runtime_endpoint.py:431

        self.pid = proc.pid

        # Before python program terminates, call shutdown implicitly. Therefore, users don't have to explicitly call .shutdown()
        atexit.register(self.shutdown)

        # Wait for server to be ready by polling /health_generate
        start_time = time.time()
        with requests.Session() as session:
            while time.time() - start_time < launch_timeout:
                try:
                    response = session.get(f"{self.url}/health_generate")
                    if response.status_code == 200:
                        break
                except requests.RequestException:
                    pass

                if not proc.is_alive():
                    self.shutdown()
                    raise RuntimeError(
                        "Initialization failed. Please see the error messages above."
                    )

                time.sleep(2)
            else:
                self.shutdown()
                raise TimeoutError("Server failed to start within the timeout period.")

        self.endpoint = RuntimeEndpoint(self.url)

    def shutdown(self):
        from sglang.srt.utils import kill_process_tree

        if self.pid is not None:
            # Note(kpham-sgl): __del__ routes here, so the reap wait has to stay
            # off -- blocking inside GC stalls whichever thread is allocating.
            kill_process_tree(self.pid, wait_timeout=None)
            self.pid = None

View on GitHub (pinned to 0132848349)

Solutions

Scroll up in the logs and read the server's own traceback — it contains the actual failure (OOM, file not found, etc.).
Fix resources: free GPU memory, pick a smaller/quantized model, correct the model path, or kill the process on the port.
Retry launch manually (sglang.launch_server ...) to iterate faster on the underlying error.

Example fix

# before
backend = sgl.RuntimeEndpoint("local", model_path="meta_llama/Llama-3-70B")  # OOM

# after
backend = sgl.RuntimeEndpoint("local", model_path="meta-llama/Llama-3-8B-Instruct")

Defensive patterns

Strategy: try-catch

Validate before calling

# preflight: GPU memory and port free
import subprocess, socket
assert subprocess.run(['nvidia-smi']).returncode == 0
with socket.socket() as s_:
    s_.bind(("0.0.0.0", port))  # raises if port busy

Try / catch

try:
    backend = sgl.RuntimeEndpoint("local", model_path=m)
except RuntimeError:
    # capture child server log, surface root cause (OOM/path), free resources and retry once
    log_child_output(); free_gpu(); retry_with_smaller_model()

Prevention

Pre-download weights and validate model path before spawn.
Run a manual launch_server once to validate args before embedding spawn in code.
Capture and persist the child server's stdout for post-mortem.

When it happens

Trigger: RuntimeEndpoint(model_path=..., spawn subprocess) where server launch fails: out of GPU memory, nonexistent/hung model download, invalid server args, or port already in use.

Common situations: Model too large for the GPU (CUDA OOM during weight load); typo'd model path/hf repo; leftover process holding the port; incompatible CUDA/torch in the environment.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e50fef03a8dc9219. Report an issue: GitHub.