sgl-project/sglang · error · ValueError

routed_dp_rank={routed_dp_rank} out of range [0, {dp_size})

Error message

routed_dp_rank={routed_dp_rank} out of range [0, {dp_size})

What it means

When data-parallel routing is enabled (dp_size > 1), each generate call may pin a request to a specific DP rank via routed_dp_rank. The rank must fall in [0, dp_size); anything else is rejected with ValueError before dispatch. routed_dp_rank=None or 0 with dp_size<=1 is silently ignored.

Source

Thrown at python/sglang/srt/entrypoints/engine.py:354

            import warnings

            warnings.warn(
                "'data_parallel_rank' is deprecated, use 'routed_dp_rank' instead.",
                DeprecationWarning,
                stacklevel=3,
            )
            if routed_dp_rank is None:
                routed_dp_rank = data_parallel_rank

        if routed_dp_rank is not None:
            dp_size = get_parallel().dp_size
            if dp_size <= 1 and routed_dp_rank == 0:
                logger.debug(
                    f"routed_dp_rank={routed_dp_rank} is ignored because dp_size={dp_size}"
                )
                return None
            if routed_dp_rank < 0 or routed_dp_rank >= dp_size:
                raise ValueError(
                    f"routed_dp_rank={routed_dp_rank} out of range [0, {dp_size})"
                )

        logger.debug(f"routed_dp_rank: {routed_dp_rank}")
        return routed_dp_rank

    def generate(
        self,
        # The input prompt. It can be a single prompt or a batch of prompts.
        prompt: Optional[Union[List[str], str]] = None,
        sampling_params: Optional[Union[List[Dict], Dict]] = None,
        # The token ids for text; one can either specify text or input_ids.
        input_ids: Optional[Union[List[List[int]], List[int]]] = None,
        # The image input. It can be an image instance, file name, URL, or base64 encoded string.
        # Can be formatted as:
        # - Single image for a single request
        # - List of images (one per request in a batch)
        # - List of lists of images (multiple images per request)

View on GitHub (pinned to 0132848349)

Solutions

1. Clamp/modulo the rank against dp_size at the call site: routed_dp_rank = user_rank % server_args.dp_size.
2. Read the rank from the engine's actual config (engine.server_args.dp_size) rather than a stale constant.
3. Pass routed_dp_rank=None (or omit it) when you don't need explicit rank pinning — the scheduler load-balances automatically.

Example fix

# before
out = engine.generate(prompts, sampling_params, routed_dp_rank=4)  # dp_size=2

# after
dp = engine.server_args.dp_size
out = engine.generate(prompts, sampling_params, routed_dp_rank=4 % dp)

dp = engine.server_args.dp_size
if routed_dp_rank is not None and not (0 <= routed_dp_rank < max(dp, 1)):
    routed_dp_rank = routed_dp_rank % dp  # or raise your own clear error
out = engine.generate(prompts, sampling_params, routed_dp_rank=routed_dp_rank)

def valid_dp_rank(rank: int | None, dp_size: int) -> bool:
    return rank is None or (isinstance(rank, int) and 0 <= rank < max(dp_size, 1))

try:
    engine.generate(prompts, sp, routed_dp_rank=r)
except ValueError as e:
    if "out of range" in str(e):
        r %= engine.server_args.dp_size
        engine.generate(prompts, sp, routed_dp_rank=r)
    else:
        raise

When it happens

Trigger: Passing routed_dp_rank=N to engine.generate(...) or engine.async_generate(...) with N < 0 or N >= server_args.dp_size (e.g. routed_dp_rank=2 with --dp-size 2).

Common situations: Hard-coding a DP rank from a previous deployment with different dp_size; computing ranks modulo the wrong parallel dimension (tp_size or ep_size instead of dp_size); loop off-by-one generating range(dp_size+1).

Related errors

• --speculative-draft-window-size must be positive, got {}.
• Invalid X-Data-Parallel-Rank header: must be an integer, got
• item_first is not supported when embeddings are supplied
• --dcp-replicate-q-proj requires --dcp-size > 1.
• --dcp-replicate-q-proj only applies to the a2a/fi_a2a DCP co

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/19023d52c82ee020. Report an issue: GitHub.