sgl-project/sglang · error · ValueError

Pipeline parallel with multiple ModuleList blocks is not sup

Error message

Pipeline parallel with multiple ModuleList blocks is not supported.

What it means

The Transformers-backend PP splitter can only handle a pp plan with a single ModuleList boundary; two or more ModuleList entries in the plan keys are ambiguous and rejected.

Source

Thrown at python/sglang/srt/models/transformers.py:861

        setattr(parent_module, child_name, module)

    def pipeline_parallel(self):
        if self.pp_group.world_size <= 1:
            return

        pp_plan = self._get_model_pp_plan()
        if not pp_plan:
            raise ValueError(
                f"{type(self.model)} does not support pipeline parallel yet!"
            )

        pp_keys = [re.sub(r"^model\.", "", name) for name in pp_plan.keys()]
        module_list_idx = None
        module_list_name = None
        for idx, name in enumerate(pp_keys):
            if isinstance(self._get_submodule_or_none(name), nn.ModuleList):
                if module_list_idx is not None:
                    raise ValueError(
                        "Pipeline parallel with multiple ModuleList blocks is not supported."
                    )
                module_list_idx = idx
                module_list_name = name

        if module_list_idx is None or module_list_name is None:
            raise ValueError(f"Could not find ModuleList in {type(self.model)}.")

        keep_prefix_modules = self.pp_group.is_first_rank or (
            getattr(self.text_config, "tie_word_embeddings", False)
            and self.pp_group.is_last_rank
        )
        for name in pp_keys[:module_list_idx]:
            if keep_prefix_modules:
                continue
            self._set_submodule(name, PPMissingLayer())
            self._register_missing_prefix(maybe_prefix("model", name))

View on GitHub (pinned to 0132848349)

Solutions

Restrict the pp plan to one ModuleList boundary for this backend
Use TP instead of PP for such models
Use a native sglang implementation supporting multi-block PP

Example fix

# before
pp_plan = {"model.encoder.layers": 1, "model.decoder.layers": 1}
# after (single boundary)
pp_plan = {"model.decoder.layers": 1}

Defensive patterns

Strategy: validation

Validate before calling

ml = [k for k in pp_plan if isinstance(model.get_submodule(k), nn.ModuleList)]
assert len(ml) <= 1, f'multiple ModuleLists: {ml}'

Prevention

Design pp plans around one layer-stack boundary
Avoid encoder-decoder models on this backend with PP

When it happens

Trigger: A model whose pp plan keys include multiple nn.ModuleList modules (e.g. separate encoder and decoder stacks).

Common situations: Encoder-decoder or MoE models with several stacked lists in the plan.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7081325d6d683b35. Report an issue: GitHub.