{"record":{"id":"52e72f60b3502300","repo":"sgl-project/sglang","slug":"initialization-failed-please-see-the-error-messag-52e72f","errorCode":null,"errorMessage":"Initialization failed. Please see the error messages above.","messagePattern":"Initialization failed\\. Please see the error messages above\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/entrypoints/engine.py","lineNumber":1787,"sourceCode":"def _wait_for_scheduler_ready(\n    scheduler_pipe_readers: List,\n    scheduler_procs: List,\n) -> List[Dict]:\n    \"\"\"Wait for the model to finish loading and return scheduler infos.\n\n    Uses poll() with timeout instead of blocking recv(), so that child process\n    death (e.g. OOM SIGKILL) is detected promptly instead of hanging forever.\n    \"\"\"\n    scheduler_infos = []\n    for i in range(len(scheduler_pipe_readers)):\n        while True:\n            if scheduler_pipe_readers[i].poll(timeout=5.0):\n                try:\n                    data = scheduler_pipe_readers[i].recv()\n                except EOFError:\n                    raise _scheduler_died_error(i, scheduler_procs[i])\n                if data[\"status\"] != \"ready\":\n                    raise RuntimeError(\n                        \"Initialization failed. Please see the error messages above.\"\n                    )\n                scheduler_infos.append(data)\n                break\n\n            # Poll timed out — check all processes for early death\n            for j in range(len(scheduler_procs)):\n                if not scheduler_procs[j].is_alive():\n                    raise _scheduler_died_error(j, scheduler_procs[j])\n\n    return scheduler_infos\n\n\ndef _calculate_rank_ranges(\n    nnodes: int, pp_size: int, tp_size: int, node_rank: int\n) -> Tuple[range, range, int, int]:\n    \"\"\"Calculate pp_rank_range and tp_rank_range for a given node.\n","sourceCodeStart":1769,"sourceCodeEnd":1805,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/entrypoints/engine.py#L1769-L1805","documentation":"After spawning scheduler subprocesses, the engine waits on pipes for each to report status \"ready\". If a scheduler sends a non-ready status, RuntimeError 'Initialization failed...' is raised — the real traceback was already printed by the scheduler subprocess above this message; this is only the parent's summary.","triggerScenarios":"Any scheduler-side startup failure: model config/download error, CUDA OOM or no visible GPUs, bad attention backend flag, tokenizer load failure. The scheduler catches it, prints the traceback to stdout/stderr, then reports non-ready to the parent which raises this.","commonSituations":"First run downloading a model to an unreachable HF endpoint; --mem-fraction-static too high causing CUDA OOM during weight load; incompatible --attention-backend flag for the model; missing GPU drivers/permissions in containers.","solutions":["Scroll up in the log to the scheduler subprocess traceback — that message points at the actual failing step (fix that first).","For CUDA OOM: lower --mem-fraction-static or use a smaller dtype/quantization.","For model issues: verify the model path/revision is reachable and the config is valid for this sglang version.","Reproduce the scheduler failure directly by running the scheduler command shown in the logs to get an untruncated traceback."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"import torch\nassert torch.cuda.is_available() and torch.cuda.device_count() >= tp_size\n# sanity: model path resolves\nassert os.path.exists(model_path) or \":\" in model_path  # local dir or repo id","typeGuard":null,"tryCatchPattern":"try:\n    engine = sgl.Engine(**kwargs)\nexcept RuntimeError as e:\n    if \"Initialization failed\" in str(e):\n        # real traceback is in scheduler stdout above; capture and re-raise a pointer\n        raise RuntimeError(\"Scheduler init failed; inspect scheduler logs above\") from e\n    raise","preventionTips":["Always read the scheduler subprocess traceback above the generic message.","Smoke-test with a tiny model + low mem-fraction first, then scale up.","Pin model revisions and validate --attention-backend compatibility per model.","Lower --mem-fraction-static when loading large models near GPU capacity."],"tags":["sglang","scheduler","startup","subprocess"],"backgroundTag":"worker-initialization-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}