sgl-project/sglang · error · ValueError

Unsupported TP style '{style}' for Transformers backend.

Error message

Unsupported TP style '{style}' for Transformers backend.

What it means

_normalize_tp_style maps common HF TP-plan style names onto the five supported strings; anything unmapped and unsupported raises, because the backend cannot shard that layer pattern.

Source

Thrown at python/sglang/srt/models/transformers.py:247

def _normalize_tp_style(style: str) -> Style:
    style = style.lower().replace("-", "_")
    style = {
        "colwiseparallel": "colwise",
        "packed_colwise": "colwise",
        "local_colwise": "colwise",
        "rowwiseparallel": "rowwise",
        "packed_rowwise": "rowwise",
        "local_rowwise": "rowwise",
        "local_packed_rowwise": "rowwise",
        "isolated": "replicate",
        "local": "replicate",
        "replicated_with_grad_allreduce": "replicate",
        "moe_tp_experts": "replicate",
    }.get(style, style)
    if style not in {"colwise", "colwise_rep", "rowwise", "rowwise_rep", "replicate"}:
        raise ValueError(f"Unsupported TP style '{style}' for Transformers backend.")
    return style


def replace_rms_norm_class(rms_norm: nn.Module, hidden_size: int) -> nn.Module:
    eps = _getattr_first(rms_norm, ("eps", "variance_epsilon"), 1e-6)
    kwargs = {"hidden_size": hidden_size, "eps": eps}
    weight_meta = getattr(rms_norm, "weight", None)
    if weight_meta is not None:
        kwargs["hidden_size"] = weight_meta.size(0)

    try:
        with torch.device("cpu"):
            weight_test = getattr(rms_norm.__class__(1), "weight", None)
    except Exception:
        weight_test = None
    is_gemma = weight_test is not None and torch.all(weight_test == 0)

    if is_gemma:

View on GitHub (pinned to 0132848349)

Solutions

1. Map/translate the unsupported style to one of colwise/colwise_rep/rowwise/rowwise_rep/replicate in _normalize_tp_style's alias dict
2. Upgrade sglang if a newer transformers style is officially supported
3. Downgrade transformers to a version whose tp_plan styles are all handled

Example fix

# before
ALIASES = {"isolated": "replicate", ...}.get(style, style)
# after (add mapping)
ALIASES = {"isolated": "replicate", "local": "replicate", "colwise_single": "colwise", ...}.get(style, style)

SUPPORTED={'colwise','colwise_rep','rowwise','rowwise_rep','replicate'}
bad=[k for k,v in tp_plan.items() if v not in SUPPORTED and v not in ALIASES]
assert not bad, bad

def normalize(s):
    return {'isolated':'replicate','local':'replicate','replicated_with_grad_allreduce':'replicate','moe_tp_experts':'replicate'}.get(s,s)

When it happens

Trigger: A tp_plan containing styles like 'sequence_parallel', 'colwise_single', or unknown custom strings after alias normalization (isolated/local/replicated_with_grad_allreduce/moe_tp_experts are mapped to replicate).

Common situations: Newer transformers versions emitting novel _tp_plan styles; hand-written tp plans with typos or unsupported layouts for this backend.

Related errors

• {type(self.model)} does not support tensor parallel yet!
• Unsupported parallel style type {type(style)}, expected str
• delta payload size mismatch: expected ${expectedSize}, got $
• This browser does not support worker image decoding
• Generate subcommand is not yet supported for model: {model_p

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ab599f8ee7c67186. Report an issue: GitHub.