sgl-project/sglang · critical · ValueError

{type(self.model)} does not support tensor parallel yet!

Error message

{type(self.model)} does not support tensor parallel yet!

What it means

For the Transformers backend, tensor parallelism requires the HF model to expose a tp plan (_tp_plan / base_model_tp_plan). With tp_size>1 and no plan, sharding is impossible so recursive_replace raises.

Source

Thrown at python/sglang/srt/models/transformers.py:762

    def _normalize_tp_plan(self, tp_plan: Mapping[str, str]) -> dict[str, Style]:
        normalized = {}
        for pattern, style in tp_plan.items():
            if pattern.startswith("^model\\."):
                pattern = "^" + pattern[len("^model\\.") :]
            elif pattern.startswith("model\\."):
                pattern = pattern[len("model\\.") :]
            elif pattern.startswith("model."):
                pattern = pattern[len("model.") :]
            normalized[pattern] = _normalize_tp_style(style)
        return normalized

    # -- Recursive module replacement (Linear + RMSNorm) --------------------
    def recursive_replace(self):
        tp_size = get_parallel().tp_size
        tp_plan = self._normalize_tp_plan(self._get_model_tp_plan())

        if not tp_plan and tp_size > 1:
            raise ValueError(
                f"{type(self.model)} does not support tensor parallel yet!"
            )

        # Prefix patterns to match from `self.model`
        prefixed_plan = {maybe_prefix("model", k): v for k, v in tp_plan.items()}

        def _recursive_replace(module: nn.Module, prefix: str):
            for child_name, child_module in module.named_children():
                qual_name = maybe_prefix(prefix, child_name)
                new_module = child_module

                if isinstance(child_module, nn.Linear):
                    pattern = next(
                        (p for p in prefixed_plan if re.match(p, qual_name)),
                        None,
                    )
                    style = prefixed_plan.get(pattern, "replicate")
                    new_module = replace_linear_class(

View on GitHub (pinned to 0132848349)

Solutions

1. Run with tp-size 1 for this model
2. Use the native sglang implementation of the architecture which has explicit TP
3. Contribute/set a _tp_plan on the HF model class so the backend can shard

Example fix

# before
--tp 8 on model without _tp_plan
# after
--tp 1 (or use native sglang impl, e.g. Qwen2ForCausalLM native path)

tp = get_parallel().tp_size
plan = getattr(HFModelCls, '_tp_plan', None) or getattr(HFModelCls, 'base_model_tp_plan', None)
assert tp == 1 or plan, 'no tp plan; run with tp=1'

When it happens

Trigger: Launching with --tp-size >1 a model whose class defines no _tp_plan attribute (only single-GPU-capable via this backend).

Common situations: Serving newly added HF architectures before they declare TP plans; assuming any model can run multi-GPU via the generic backend.

Related errors

• Unsupported TP style '{style}' for Transformers backend.
• Unsupported parallel style type {type(style)}, expected str
• delta payload size mismatch: expected ${expectedSize}, got $
• This browser does not support worker image decoding
• Generate subcommand is not yet supported for model: {model_p

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4150d3eacb0c2eea. Report an issue: GitHub.