What causes "Role {role_type.value} rank {rank_idx} failed to initialize." in sgl-project/sglang?

Check per-rank worker logs/stack traces for the underlying exception on the reported rank.

sgl-project/sglang · critical · RuntimeError

Role {role_type.value} rank {rank_idx} failed to initialize.

Error message

Role {role_type.value} rank {rank_idx} failed to initialize.

What it means

Each disagg role process spawns GPU worker subprocesses and reads a readiness status over a pipe/socket. If a worker reports a status other than "ready" (or the earlier code path logs 'is dead' and re-raises), the launcher raises 'Role {role} rank {rank} failed to initialize', meaning a per-rank worker failed during model initialization (CUDA init, weight loading, NCCL setup).

Source

Thrown at python/sglang/multimodal_gen/runtime/launch_server.py:774

            daemon=True,
        )
        process.start()
        processes.append(process)
        readers.append(reader)

    # Wait for all ranks to be ready (after all are spawned)
    for rank_idx, reader in enumerate(readers):
        try:
            data = reader.recv()
        except EOFError:
            logger.error(
                "Role %s rank %d is dead.",
                role_type.value,
                rank_idx,
            )
            raise
        if data.get("status") != "ready":
            raise RuntimeError(
                f"Role {role_type.value} rank {rank_idx} failed to initialize."
            )
        reader.close()

    logger.info(
        "Role %s ready (%d GPU(s), work=%s)",
        role_type.value.upper(),
        num_gpus,
        work_endpoint,
    )

    # Block until interrupted
    try:
        for p in processes:
            p.join()
    except KeyboardInterrupt:
        logger.info("Role %s shutting down.", role_type.value)
    finally:

View on GitHub (pinned to 0132848349)

Solutions

Check per-rank worker logs/stack traces for the underlying exception on the reported rank.
Run `nvidia-smi` to confirm all ranks' GPUs are visible and have enough free memory for the model + TP shards.
Verify tensor-parallel size divides the available GPUs and NCCL env (e.g. master port) is set consistently across ranks.
Re-run with a smaller TP size or smaller model to isolate memory vs setup failures.

Defensive patterns

Strategy: validation

Validate before calling

import torch
total = torch.cuda.device_count()
assert server_args.tp_size <= total, f"tp={server_args.tp_size} > visible GPUs={total}"
free = [torch.cuda.mem_get_info(i)[0] for i in range(total)]
assert min(free) > MIN_BYTES_NEEDED, "not enough free VRAM on at least one rank"

Try / catch

try:
    launch_disagg_role(server_args)
except RuntimeError as e:
    if "failed to initialize" in str(e):
        # per-rank logs contain the root cause; retry after fixing memory/NCCL
        raise

Prevention

Pre-flight check GPU count/free VRAM per rank before launch.
Pin NCCL socket/master settings consistently in multi-process scripts.
Capture each worker's stderr separately so rank failures are attributable.

When it happens

Trigger: launch_disagg_role spawning N GPU workers; one rank crashes or reports status != 'ready' — e.g. CUDA OOM on one GPU, NCCL mismatch with tensor-parallel size, or an exception inside the worker before it sends the ready signal.

Common situations: Heterogeneous GPUs where one device has less free memory; tp/size larger than visible GPUs; NCCL/IPC issues in containers; one rank importing a missing dependency. The rank index in the message tells you which worker to inspect.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/061a349d979bdb82. Report an issue: GitHub.