sgl-project/sglang · error · ValueError

Could not find ModuleList in {type(self.model)}.

Error message

Could not find ModuleList in {type(self.model)}.

What it means

Mirror of 5631: pipeline_parallel needs at least one ModuleList among the pp plan keys to know where to split layers; none found means the plan doesn't reference a layer stack.

Source

Thrown at python/sglang/srt/models/transformers.py:868

        if not pp_plan:
            raise ValueError(
                f"{type(self.model)} does not support pipeline parallel yet!"
            )

        pp_keys = [re.sub(r"^model\.", "", name) for name in pp_plan.keys()]
        module_list_idx = None
        module_list_name = None
        for idx, name in enumerate(pp_keys):
            if isinstance(self._get_submodule_or_none(name), nn.ModuleList):
                if module_list_idx is not None:
                    raise ValueError(
                        "Pipeline parallel with multiple ModuleList blocks is not supported."
                    )
                module_list_idx = idx
                module_list_name = name

        if module_list_idx is None or module_list_name is None:
            raise ValueError(f"Could not find ModuleList in {type(self.model)}.")

        keep_prefix_modules = self.pp_group.is_first_rank or (
            getattr(self.text_config, "tie_word_embeddings", False)
            and self.pp_group.is_last_rank
        )
        for name in pp_keys[:module_list_idx]:
            if keep_prefix_modules:
                continue
            self._set_submodule(name, PPMissingLayer())
            self._register_missing_prefix(maybe_prefix("model", name))

        layers = self.model.get_submodule(module_list_name)
        self.start_layer, self.end_layer = get_pp_indices(
            len(layers),
            self.pp_group.rank_in_group,
            self.pp_group.world_size,
        )
        for idx in range(len(layers)):

View on GitHub (pinned to 0132848349)

Solutions

1. Add the decoder layer ModuleList path to the pp plan keys
2. Check the module name via named_modules() and correct the plan key
3. Use pp-size 1 or a native implementation

Example fix

# before
pp_plan = {"model.embed_tokens": 1}
# after
pp_plan = {"model.embed_tokens": 1, "model.layers": "loop"}

ml = [k for k in pp_plan if isinstance(model.get_submodule(k), nn.ModuleList)]
assert ml, 'pp plan has no ModuleList to split on'

When it happens

Trigger: A pp plan whose keys are all plain modules (embeddings, norms) with no layers ModuleList.

Common situations: Hand-written or auto-derived pp plans omitting the decoder layers path; renamed layer module (e.g. 'blocks' vs 'layers') not in the plan keys.

Related errors

• {type(self.model)} does not support pipeline parallel yet!
• Pipeline parallel with multiple ModuleList blocks is not sup
• Currently DFLASH speculative decoding only supports pp_size
• Currently DSpark speculative decoding only supports pp_size
• Unexpected compressed-MLA dst_kv_ptrs length {len(dst_kv_ptr

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/98321adb0df9d25f. Report an issue: GitHub.