sgl-project/sglang · error · ValueError

Vision tp_size and tp_rank must be set together

Error message

Vision tp_size and tp_rank must be set together

What it means

When overriding Qwen3-VL vision tensor parallelism you must supply both tp_size and tp_rank; supplying only one is rejected because the pair is meaningless alone.

Source

Thrown at python/sglang/srt/models/qwen3_vl.py:116

_is_cpu = is_cpu()

# Below this image count the per-image loop beats the vectorized path (which has a
# fixed setup cost; measured crossover ~6 on H20); both give the same result.
_VECTORIZED_VL_POS_EMBED_MIN_IMAGES = 6


def _resolve_vision_tp(
    *,
    use_data_parallel: bool,
    tp_size: Optional[int],
    tp_rank: Optional[int],
) -> tuple[int, int]:
    if use_data_parallel:
        if tp_size is not None or tp_rank is not None:
            raise ValueError("Explicit vision TP cannot be combined with data parallel")
        return 1, 0
    if (tp_size is None) != (tp_rank is None):
        raise ValueError("Vision tp_size and tp_rank must be set together")
    if tp_size is None:
        parallel = get_parallel()
        return parallel.attn_tp_size, parallel.attn_tp_rank
    assert tp_rank is not None
    return tp_size, tp_rank


class Qwen3_VisionMLP(nn.Module):

    def __init__(
        self,
        in_features: int,
        hidden_features: int,
        bias: bool = True,
        hidden_act="silu",
        quant_config: Optional[QuantizationConfig] = None,
        prefix: str = "",
        use_data_parallel: bool = False,

View on GitHub (pinned to 0132848349)

Solutions

1. Pass both tp_size and tp_rank together
2. Or pass neither to fall back to attn_tp_size/attn_tp_rank from get_parallel()

Example fix

# before
_resolve_vision_tp(use_data_parallel=False, tp_size=4, tp_rank=None)
# after
_resolve_vision_tp(use_data_parallel=False, tp_size=4, tp_rank=0)

assert (tp_size is None) == (tp_rank is None), "set both or neither"

def valid_vision_tp(tp_size, tp_rank) -> bool:
    return (tp_size is None and tp_rank is None) or (tp_size is not None and tp_rank is not None)

When it happens

Trigger: Constructing the vision tower with tp_size set but tp_rank None (or vice versa).

Common situations: Custom integration code partially specifying vision TP; forgetting the rank when wiring a distributed launcher.

Related errors

• Explicit vision TP cannot be combined with data parallel
• delta payload size mismatch: expected ${expectedSize}, got $
• This browser does not support worker image decoding
• Generate subcommand is not yet supported for model: {model_p
• Error: --model-type requires a non-empty value.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2867c829a1fe2148. Report an issue: GitHub.