sgl-project/sglang · critical · RuntimeError

Initialization failed. Please see the error messages above.

Error message

Initialization failed. Please see the error messages above.

What it means

After spawning scheduler subprocesses, the engine waits on pipes for each to report status "ready". If a scheduler sends a non-ready status, RuntimeError 'Initialization failed...' is raised — the real traceback was already printed by the scheduler subprocess above this message; this is only the parent's summary.

Source

Thrown at python/sglang/srt/entrypoints/engine.py:1787

def _wait_for_scheduler_ready(
    scheduler_pipe_readers: List,
    scheduler_procs: List,
) -> List[Dict]:
    """Wait for the model to finish loading and return scheduler infos.

    Uses poll() with timeout instead of blocking recv(), so that child process
    death (e.g. OOM SIGKILL) is detected promptly instead of hanging forever.
    """
    scheduler_infos = []
    for i in range(len(scheduler_pipe_readers)):
        while True:
            if scheduler_pipe_readers[i].poll(timeout=5.0):
                try:
                    data = scheduler_pipe_readers[i].recv()
                except EOFError:
                    raise _scheduler_died_error(i, scheduler_procs[i])
                if data["status"] != "ready":
                    raise RuntimeError(
                        "Initialization failed. Please see the error messages above."
                    )
                scheduler_infos.append(data)
                break

            # Poll timed out — check all processes for early death
            for j in range(len(scheduler_procs)):
                if not scheduler_procs[j].is_alive():
                    raise _scheduler_died_error(j, scheduler_procs[j])

    return scheduler_infos


def _calculate_rank_ranges(
    nnodes: int, pp_size: int, tp_size: int, node_rank: int
) -> Tuple[range, range, int, int]:
    """Calculate pp_rank_range and tp_rank_range for a given node.

View on GitHub (pinned to 0132848349)

Solutions

Scroll up in the log to the scheduler subprocess traceback — that message points at the actual failing step (fix that first).
For CUDA OOM: lower --mem-fraction-static or use a smaller dtype/quantization.
For model issues: verify the model path/revision is reachable and the config is valid for this sglang version.
Reproduce the scheduler failure directly by running the scheduler command shown in the logs to get an untruncated traceback.

Defensive patterns

Strategy: fallback

Validate before calling

import torch
assert torch.cuda.is_available() and torch.cuda.device_count() >= tp_size
# sanity: model path resolves
assert os.path.exists(model_path) or ":" in model_path  # local dir or repo id

Try / catch

try:
    engine = sgl.Engine(**kwargs)
except RuntimeError as e:
    if "Initialization failed" in str(e):
        # real traceback is in scheduler stdout above; capture and re-raise a pointer
        raise RuntimeError("Scheduler init failed; inspect scheduler logs above") from e
    raise

Prevention

Always read the scheduler subprocess traceback above the generic message.
Smoke-test with a tiny model + low mem-fraction first, then scale up.
Pin model revisions and validate --attention-backend compatibility per model.
Lower --mem-fraction-static when loading large models near GPU capacity.

When it happens

Trigger: Any scheduler-side startup failure: model config/download error, CUDA OOM or no visible GPUs, bad attention backend flag, tokenizer load failure. The scheduler catches it, prints the traceback to stdout/stderr, then reports non-ready to the parent which raises this.

Common situations: First run downloading a model to an unreachable HF endpoint; --mem-fraction-static too high causing CUDA OOM during weight load; incompatible --attention-backend flag for the model; missing GPU drivers/permissions in containers.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/52e72f60b3502300. Report an issue: GitHub.