{"record":{"id":"4150d3eacb0c2eea","repo":"sgl-project/sglang","slug":"type-self-model-does-not-support-tensor-paralle","errorCode":null,"errorMessage":"{type(self.model)} does not support tensor parallel yet!","messagePattern":"(.+?) does not support tensor parallel yet!","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/models/transformers.py","lineNumber":762,"sourceCode":"    def _normalize_tp_plan(self, tp_plan: Mapping[str, str]) -> dict[str, Style]:\n        normalized = {}\n        for pattern, style in tp_plan.items():\n            if pattern.startswith(\"^model\\\\.\"):\n                pattern = \"^\" + pattern[len(\"^model\\\\.\") :]\n            elif pattern.startswith(\"model\\\\.\"):\n                pattern = pattern[len(\"model\\\\.\") :]\n            elif pattern.startswith(\"model.\"):\n                pattern = pattern[len(\"model.\") :]\n            normalized[pattern] = _normalize_tp_style(style)\n        return normalized\n\n    # -- Recursive module replacement (Linear + RMSNorm) --------------------\n    def recursive_replace(self):\n        tp_size = get_parallel().tp_size\n        tp_plan = self._normalize_tp_plan(self._get_model_tp_plan())\n\n        if not tp_plan and tp_size > 1:\n            raise ValueError(\n                f\"{type(self.model)} does not support tensor parallel yet!\"\n            )\n\n        # Prefix patterns to match from `self.model`\n        prefixed_plan = {maybe_prefix(\"model\", k): v for k, v in tp_plan.items()}\n\n        def _recursive_replace(module: nn.Module, prefix: str):\n            for child_name, child_module in module.named_children():\n                qual_name = maybe_prefix(prefix, child_name)\n                new_module = child_module\n\n                if isinstance(child_module, nn.Linear):\n                    pattern = next(\n                        (p for p in prefixed_plan if re.match(p, qual_name)),\n                        None,\n                    )\n                    style = prefixed_plan.get(pattern, \"replicate\")\n                    new_module = replace_linear_class(","sourceCodeStart":744,"sourceCodeEnd":780,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/transformers.py#L744-L780","documentation":"For the Transformers backend, tensor parallelism requires the HF model to expose a tp plan (_tp_plan / base_model_tp_plan). With tp_size>1 and no plan, sharding is impossible so recursive_replace raises.","triggerScenarios":"Launching with --tp-size >1 a model whose class defines no _tp_plan attribute (only single-GPU-capable via this backend).","commonSituations":"Serving newly added HF architectures before they declare TP plans; assuming any model can run multi-GPU via the generic backend.","solutions":["Run with tp-size 1 for this model","Use the native sglang implementation of the architecture which has explicit TP","Contribute/set a _tp_plan on the HF model class so the backend can shard"],"exampleFix":"# before\n--tp 8 on model without _tp_plan\n# after\n--tp 1 (or use native sglang impl, e.g. Qwen2ForCausalLM native path)","handlingStrategy":"validation","validationCode":"tp = get_parallel().tp_size\nplan = getattr(HFModelCls, '_tp_plan', None) or getattr(HFModelCls, 'base_model_tp_plan', None)\nassert tp == 1 or plan, 'no tp plan; run with tp=1'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check for _tp_plan before multi-GPU launches","Use architecture-native implementations for TP"],"tags":["tensor-parallel","transformers-backend","tp-plan"],"backgroundTag":"no-tp-plan-for-model","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}