sgl-project/sglang · error · RuntimeError

world_size ({world_size}) is not equal to tensor_model_paral

Error message

world_size ({world_size}) is not equal to tensor_model_parallel_size ({tensor_model_parallel_size}) x pipeline_model_parallel_size ({pipeline_model_parallel_size})

What it means

initialize_model_parallel validates that world_size == tensor_model_parallel_size * pipeline_model_parallel_size (using the torch.distributed world size, or the recovered rank's world size when recovering). TP and PP must exactly tile the total number of ranks; any remainder is a configuration error.

Source

Thrown at python/sglang/srt/distributed/parallel_state.py:2415

    Note that for efficiency, the caller should make sure adjacent ranks
    are on the same DGX box. For example if we are using 2 DGX-1 boxes
    with a total of 16 GPUs, rank 0 to 7 belong to the first box and
    ranks 8 to 15 belong to the second box.
    """
    # Get world size and rank. Ensure some consistencies.
    assert torch.distributed.is_initialized()
    backend = backend or torch.distributed.get_backend(get_world_group().device_group)

    # Joiners construct their local TP/PP layout in global rank space.
    world_size: int = (
        tensor_model_parallel_size * pipeline_model_parallel_size
        if recovered_rank
        else torch.distributed.get_world_size()
    )

    if world_size != tensor_model_parallel_size * pipeline_model_parallel_size:
        raise RuntimeError(
            f"world_size ({world_size}) is not equal to "
            f"tensor_model_parallel_size ({tensor_model_parallel_size}) x "
            f"pipeline_model_parallel_size ({pipeline_model_parallel_size})"
        )
    if decode_context_parallel_size < 1:
        raise RuntimeError(
            f"decode_context_parallel_size ({decode_context_parallel_size}) must be >= 1"
        )
    if decode_context_parallel_size > 1 and not (is_hip() or is_cuda()):
        raise RuntimeError(
            "Decode context parallel (decode_context_parallel_size > 1) is "
            "currently only supported on the AMD HIP platform or CUDA platform, but got "
            f"decode_context_parallel_size ({decode_context_parallel_size}) "
            "on a non-HIP or non-CUDA platform."
        )
    if tensor_model_parallel_size % decode_context_parallel_size != 0:
        raise RuntimeError(
            f"tensor_model_parallel_size ({tensor_model_parallel_size}) must be divisible by "

View on GitHub (pinned to 0132848349)

Solutions

  1. Make tp_size * pp_size equal the number of distributed ranks, e.g. 8 GPUs -> --tp 8 --pp 1, or --tp 4 --pp 2
  2. Check torch.distributed.get_world_size() / number of processes (rays/CPU workers excluded appropriately) and adjust launch flags
  3. If you intended data parallelism, configure the DP dimension so that the overall rank count matches the product per replica

Example fix

# before (8 GPUs)
python -m sglang.launch_server --model M --tp-size 4 --pp-size 1
# RuntimeError: world_size (8) is not equal to 4 x 1

# after
python -m sglang.launch_server --model M --tp-size 8 --pp-size 1
# or
python -m sglang.launch_server --model M --tp-size 4 --pp-size 2
Defensive patterns

Strategy: validation

Validate before calling

import torch
ws = torch.distributed.get_world_size()
assert ws == tp_size * pp_size, f'{ws} != {tp_size}x{pp_size}; fix --tp/--pp or GPU count'

Try / catch

try:
    initialize_model_parallel(tensor_model_parallel_size=tp, pipeline_model_parallel_size=pp)
except RuntimeError as e:
    if 'world_size' in str(e):
        raise SystemExit(f'Adjust tp*pp to world size: {e}') from e
    raise

Prevention

When it happens

Trigger: Calling initialize_model_parallel (via _init_parallel_groups or ensure_model_parallel_initialized) with --tp-size and --pp-size whose product differs from the total world size (number of GPUs/processes in the torch.distributed job), e.g. tp=4, pp=2 with 8 world size mismatch (needs 8 = 4*2; fails for tp=4, pp=1 on 8 GPUs when DP is not accounted for).

Common situations: Launching on a different GPU count than the script assumes (e.g. 8-GPU launch args on a 4-GPU node); forgetting that data-parallel/attention-EP ranks must be included in world_size; changing --tp-size without adjusting --pp-size or GPU count; recovery paths with a stale world size.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8478823282da8f58. Report an issue: GitHub.