sgl-project/sglang · error · RuntimeError

Decode context parallel (decode_context_parallel_size > 1) i

Error message

Decode context parallel (decode_context_parallel_size > 1) is currently only supported on the AMD HIP platform or CUDA platform, but got decode_context_parallel_size ({decode_context_parallel_size}) on a non-HIP or non-CUDA platform.

What it means

Decode context parallelism (decode_context_parallel_size > 1) splits decode-phase attention across ranks and its kernels are only implemented for AMD HIP and CUDA platforms. initialize_model_parallel rejects DCP > 1 on any other platform (e.g. CPU or other accelerators).

Source

Thrown at python/sglang/srt/distributed/parallel_state.py:2425

    # Joiners construct their local TP/PP layout in global rank space.
    world_size: int = (
        tensor_model_parallel_size * pipeline_model_parallel_size
        if recovered_rank
        else torch.distributed.get_world_size()
    )

    if world_size != tensor_model_parallel_size * pipeline_model_parallel_size:
        raise RuntimeError(
            f"world_size ({world_size}) is not equal to "
            f"tensor_model_parallel_size ({tensor_model_parallel_size}) x "
            f"pipeline_model_parallel_size ({pipeline_model_parallel_size})"
        )
    if decode_context_parallel_size < 1:
        raise RuntimeError(
            f"decode_context_parallel_size ({decode_context_parallel_size}) must be >= 1"
        )
    if decode_context_parallel_size > 1 and not (is_hip() or is_cuda()):
        raise RuntimeError(
            "Decode context parallel (decode_context_parallel_size > 1) is "
            "currently only supported on the AMD HIP platform or CUDA platform, but got "
            f"decode_context_parallel_size ({decode_context_parallel_size}) "
            "on a non-HIP or non-CUDA platform."
        )
    if tensor_model_parallel_size % decode_context_parallel_size != 0:
        raise RuntimeError(
            f"tensor_model_parallel_size ({tensor_model_parallel_size}) must be divisible by "
            f"decode_context_parallel_size ({decode_context_parallel_size})"
        )

    # Build the tensor model-parallel groups.
    num_tensor_model_parallel_groups: int = world_size // tensor_model_parallel_size
    global _TP
    assert _TP is None, "tensor model parallel group is already initialized"
    group_ranks = []
    for tp_group_idx in range(num_tensor_model_parallel_groups):
        ranks = list(

View on GitHub (pinned to 0132848349)

Solutions

1. Install/use a CUDA or ROCm build of PyTorch and GPU hardware so is_cuda()/is_hip() is true
2. If on unsupported hardware, set decode_context_parallel_size back to 1
3. Verify platform detection: python -c 'import torch; print(torch.cuda.is_available(), torch.version.hip)'

Example fix

# before (CPU-only torch)
--decode-context-parallel-size 2
# after: install CUDA torch and keep the flag, or run without DCP
pip install torch --index-url https://download.pytorch.org/whl/cu121
# or
--decode-context-parallel-size 1

from sglang.srt.utils import is_cuda, is_hip
if dcp_size > 1:
    assert is_cuda() or is_hip(), 'DCP>1 requires CUDA or HIP platform'

def dcp_supported() -> bool:
    from sglang.srt.utils import is_cuda, is_hip
    return is_cuda() or is_hip()

When it happens

Trigger: Calling initialize_model_parallel with decode_context_parallel_size > 1 while is_hip() and is_cuda() are both false — i.e. running on a non-CUDA/non-HIP device or torch build.

Common situations: Reusing a DCP-enabled launch config on CPU/other-accelerator machines (CI, dev laptops); running a CPU-only torch build (torch.version.cuda is None makes is_cuda() false) on GPU hardware by accident.

Related errors

• decode_context_parallel_size ({decode_context_parallel_size}
• tensor_model_parallel_size ({tensor_model_parallel_size}) mu
• HiSparse device KV transfer requires sgl_kernel.kvcacheio (C
• KV4 is not tested on non-CUDA platforms.
• HIP does not support fused_marlin_moe currently.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/db5e8e90c8a93636. Report an issue: GitHub.