{"record":{"id":"db8dd8951153c1a3","repo":"sgl-project/sglang","slug":"intern-s2-mobius-baseline-does-not-support-pipelin","errorCode":null,"errorMessage":"Intern-S2-Mobius baseline does not support pipeline parallelism","messagePattern":"Intern-S2-Mobius baseline does not support pipeline parallelism","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/interns2_mobius.py","lineNumber":700,"sourceCode":"            )\n        return self._forward_after_attention(\n            hidden_states, residual, forward_batch, meta_mlp\n        )\n\n\nclass InternS2MobiusForCausalLM(Qwen3_5ForCausalLM):\n    def __init__(\n        self,\n        config: InternS2MobiusTextConfig,\n        quant_config: QuantizationConfig | None = None,\n        prefix: str = \"\",\n    ) -> None:\n        nn.Module.__init__(self)\n        self.config = config\n        self.hidden_size = config.hidden_size\n        self.pp_group = get_pp_group()\n        if self.pp_group.world_size != 1:\n            raise ValueError(\n                \"Intern-S2-Mobius baseline does not support pipeline parallelism\"\n            )\n\n        alt_stream = get_stream(\"alt\") if _is_cuda else None\n        self.embed_tokens = VocabParallelEmbedding(\n            config.vocab_size,\n            config.hidden_size,\n            org_num_embeddings=config.vocab_size,\n            enable_tp=not is_dp_attention_enabled(),\n        )\n\n        bank_prefix = prefix.replace(\"model.language_model\", \"model\")\n        self.meta_mlp = nn.ModuleList(\n            [\n                InternS2MobiusRoutedExpertBank(\n                    bank_id=bank_id,\n                    config=config,\n                    quant_config=quant_config,","sourceCodeStart":682,"sourceCodeEnd":718,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/interns2_mobius.py#L682-L718","documentation":"The baseline InternS2MobiusModel explicitly refuses pipeline parallelism (pp world_size != 1). The baseline implementation has no inter-stage communication; the conditional-generation wrapper handles layer partitioning.","triggerScenarios":"Launching with --pipeline-parallel-size > 1 (or a PP env) while the runtime instantiates the baseline model class directly.","commonSituations":"Multi-node serving with PP, or copied launch scripts that set PP flags. Using InternS2MobiusForConditionalGeneration instead routes around this path.","solutions":["Drop --pipeline-parallel-size / set it to 1 and scale with TP instead","Load the model via the InternS2MobiusForConditionalGeneration wrapper rather than the baseline class","If PP is required, extend the model with pp_group support upstream"],"exampleFix":"# before\n--pp 2 --tp 4\n\n# after\n--pp 1 --tp 8","handlingStrategy":"validation","validationCode":"from sglang.srt.distributed import get_parallel_state\nassert get_parallel_state().get_pp_group().world_size == 1, \"baseline model requires PP=1\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep pipeline-parallel flags off for the baseline model","Use the ConditionalGeneration wrapper for PP deployments"],"tags":["pipeline-parallel","unsupported-feature","launch-config"],"backgroundTag":"unsupported-parallelism-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}