sgl-project/sglang · error · RuntimeError

tensor_model_parallel_size ({tensor_model_parallel_size}) mu

Error message

tensor_model_parallel_size ({tensor_model_parallel_size}) must be divisible by decode_context_parallel_size ({decode_context_parallel_size})

What it means

With decode context parallelism enabled, each TP group is partitioned into DCP subgroups, so tensor_model_parallel_size must be an exact multiple of decode_context_parallel_size. initialize_model_parallel enforces this with a modulo check before building tensor-model-parallel groups.

Source

Thrown at python/sglang/srt/distributed/parallel_state.py:2432

    if world_size != tensor_model_parallel_size * pipeline_model_parallel_size:
        raise RuntimeError(
            f"world_size ({world_size}) is not equal to "
            f"tensor_model_parallel_size ({tensor_model_parallel_size}) x "
            f"pipeline_model_parallel_size ({pipeline_model_parallel_size})"
        )
    if decode_context_parallel_size < 1:
        raise RuntimeError(
            f"decode_context_parallel_size ({decode_context_parallel_size}) must be >= 1"
        )
    if decode_context_parallel_size > 1 and not (is_hip() or is_cuda()):
        raise RuntimeError(
            "Decode context parallel (decode_context_parallel_size > 1) is "
            "currently only supported on the AMD HIP platform or CUDA platform, but got "
            f"decode_context_parallel_size ({decode_context_parallel_size}) "
            "on a non-HIP or non-CUDA platform."
        )
    if tensor_model_parallel_size % decode_context_parallel_size != 0:
        raise RuntimeError(
            f"tensor_model_parallel_size ({tensor_model_parallel_size}) must be divisible by "
            f"decode_context_parallel_size ({decode_context_parallel_size})"
        )

    # Build the tensor model-parallel groups.
    num_tensor_model_parallel_groups: int = world_size // tensor_model_parallel_size
    global _TP
    assert _TP is None, "tensor model parallel group is already initialized"
    group_ranks = []
    for tp_group_idx in range(num_tensor_model_parallel_groups):
        ranks = list(
            range(
                tp_group_idx * tensor_model_parallel_size,
                (tp_group_idx + 1) * tensor_model_parallel_size,
            )
        )
        group_ranks.append(ranks)

View on GitHub (pinned to 0132848349)

Solutions

Pick decode_context_parallel_size that divides tensor_model_parallel_size (e.g. tp=8: dcp in {1,2,4,8})
Or raise/lower tp_size to a multiple of the desired DCP size
Add a startup assertion in your launcher: assert tp % dcp == 0

Example fix

# before
--tp-size 4 --decode-context-parallel-size 3
# after
--tp-size 4 --decode-context-parallel-size 2
# or
--tp-size 6 --decode-context-parallel-size 3

Defensive patterns

Strategy: validation

Validate before calling

assert tp_size % decode_context_parallel_size == 0, \
    f'tp={tp_size} must be divisible by dcp={decode_context_parallel_size}'

Try / catch

try:
    initialize_model_parallel(tensor_model_parallel_size=tp,
                             decode_context_parallel_size=dcp)
except RuntimeError as e:
    if 'divisible' in str(e):
        dcp = max(d for d in range(1, tp + 1) if tp % d == 0 and d <= dcp)
        initialize_model_parallel(tensor_model_parallel_size=tp,
                                 decode_context_parallel_size=dcp)
    else:
        raise

Prevention

Choose dcp as a power-of-2 divisor of tp (tp//2, tp//4, ...)
Add launcher assertion tp % dcp == 0
Update both flags together when tuning parallelism

When it happens

Trigger: Calling initialize_model_parallel with tensor_model_parallel_size % decode_context_parallel_size != 0, e.g. tp=4 with decode_context_parallel_size=3.

Common situations: Independently tuning --tp-size and --decode-context-parallel-size without keeping divisibility; template configs generated per-node GPU count producing odd pairs; changing tp for a model while forgetting to update the DCP setting.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/981b8f6510830296. Report an issue: GitHub.