{"record":{"id":"ab599f8ee7c67186","repo":"sgl-project/sglang","slug":"unsupported-tp-style-style-for-transformers-ba","errorCode":null,"errorMessage":"Unsupported TP style '{style}' for Transformers backend.","messagePattern":"Unsupported TP style '(.+?)' for Transformers backend\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/transformers.py","lineNumber":247,"sourceCode":"\n\ndef _normalize_tp_style(style: str) -> Style:\n    style = style.lower().replace(\"-\", \"_\")\n    style = {\n        \"colwiseparallel\": \"colwise\",\n        \"packed_colwise\": \"colwise\",\n        \"local_colwise\": \"colwise\",\n        \"rowwiseparallel\": \"rowwise\",\n        \"packed_rowwise\": \"rowwise\",\n        \"local_rowwise\": \"rowwise\",\n        \"local_packed_rowwise\": \"rowwise\",\n        \"isolated\": \"replicate\",\n        \"local\": \"replicate\",\n        \"replicated_with_grad_allreduce\": \"replicate\",\n        \"moe_tp_experts\": \"replicate\",\n    }.get(style, style)\n    if style not in {\"colwise\", \"colwise_rep\", \"rowwise\", \"rowwise_rep\", \"replicate\"}:\n        raise ValueError(f\"Unsupported TP style '{style}' for Transformers backend.\")\n    return style\n\n\ndef replace_rms_norm_class(rms_norm: nn.Module, hidden_size: int) -> nn.Module:\n    eps = _getattr_first(rms_norm, (\"eps\", \"variance_epsilon\"), 1e-6)\n    kwargs = {\"hidden_size\": hidden_size, \"eps\": eps}\n    weight_meta = getattr(rms_norm, \"weight\", None)\n    if weight_meta is not None:\n        kwargs[\"hidden_size\"] = weight_meta.size(0)\n\n    try:\n        with torch.device(\"cpu\"):\n            weight_test = getattr(rms_norm.__class__(1), \"weight\", None)\n    except Exception:\n        weight_test = None\n    is_gemma = weight_test is not None and torch.all(weight_test == 0)\n\n    if is_gemma:","sourceCodeStart":229,"sourceCodeEnd":265,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/transformers.py#L229-L265","documentation":"_normalize_tp_style maps common HF TP-plan style names onto the five supported strings; anything unmapped and unsupported raises, because the backend cannot shard that layer pattern.","triggerScenarios":"A tp_plan containing styles like 'sequence_parallel', 'colwise_single', or unknown custom strings after alias normalization (isolated/local/replicated_with_grad_allreduce/moe_tp_experts are mapped to replicate).","commonSituations":"Newer transformers versions emitting novel _tp_plan styles; hand-written tp plans with typos or unsupported layouts for this backend.","solutions":["Map/translate the unsupported style to one of colwise/colwise_rep/rowwise/rowwise_rep/replicate in _normalize_tp_style's alias dict","Upgrade sglang if a newer transformers style is officially supported","Downgrade transformers to a version whose tp_plan styles are all handled"],"exampleFix":"# before\nALIASES = {\"isolated\": \"replicate\", ...}.get(style, style)\n# after (add mapping)\nALIASES = {\"isolated\": \"replicate\", \"local\": \"replicate\", \"colwise_single\": \"colwise\", ...}.get(style, style)","handlingStrategy":"validation","validationCode":"SUPPORTED={'colwise','colwise_rep','rowwise','rowwise_rep','replicate'}\nbad=[k for k,v in tp_plan.items() if v not in SUPPORTED and v not in ALIASES]\nassert not bad, bad","typeGuard":"def normalize(s):\n    return {'isolated':'replicate','local':'replicate','replicated_with_grad_allreduce':'replicate','moe_tp_experts':'replicate'}.get(s,s)","tryCatchPattern":null,"preventionTips":["Pin transformers version tested with sglang","Extend the alias map when adopting new transformers releases"],"tags":["tensor-parallel","transformers-backend","tp-plan"],"backgroundTag":"unsupported-tp-plan-style","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}