sgl-project/sglang · critical · ValueError

num_nextn_predict_layers is not in the config

Error message

num_nextn_predict_layers is not in the config

What it means

When loading weights, bailing_moe expects speculative-MTP (nextn) layer info: either config already contains the layer indices or it must define num_nextn_predict_layers so the nextn layer id can be derived. If neither is present, load_weights raises.

Source

Thrown at python/sglang/srt/models/bailing_moe.py:887

            return self.logits_processor(
                input_ids, hidden_states, self.lm_head, forward_batch, aux_hidden_states
            )
        else:
            return hidden_states

    def load_weights(self, weights: Iterable[Tuple[str, torch.Tensor]], is_nextn=False):
        if is_nextn:
            if hasattr(self.config, "num_nextn_predict_layers"):
                num_nextn_layers = self.config.num_nextn_predict_layers
                assert num_nextn_layers == 1, "Only 1 nextn layer is supported"
                # compatible with old design
                nextn_layer_id = (
                    0
                    if self.config.num_hidden_layers == 1
                    else self.config.num_hidden_layers
                )
            else:
                raise ValueError("num_nextn_predict_layers is not in the config")

        stacked_params_mapping = [
            # (param_name, shard_name, shard_id)
            ("gate_up_proj", "gate_proj", 0),
            ("gate_up_proj", "up_proj", 1),
        ]

        if is_nextn:
            nextn_layer_prefix = f"model.layers.{nextn_layer_id}"
            nextn_spec_weight_names = [
                "final_layernorm",
                "eh_proj",
                "enorm",
                "hnorm",
            ]
        # Params for weights, fp8 weight scales, fp8 activation scales
        # (param_name, weight_name, expert_id, shard_id)
        expert_params_mapping = FusedMoE.make_expert_params_mapping(

View on GitHub (pinned to 0132848349)

Solutions

1. Add "num_nextn_predict_layers": 1 to config.json (only 1 nextn layer is supported)
2. Or ensure the config carries the explicit nextn layer index fields the loader checks before this branch
3. Disable speculative/MTP decoding if the checkpoint has no nextn weights

Example fix

// before
{ "num_hidden_layers": 60 }
// after
{ "num_hidden_layers": 60, "num_nextn_predict_layers": 1 }

assert hasattr(config, "num_nextn_predict_layers"), "add num_nextn_predict_layers=1 for MTP"

When it happens

Trigger: Loading a Bailing MoE checkpoint in MTP/speculative mode (or a checkpoint containing nextn weights) when config.json lacks num_nextn_predict_layers.

Common situations: Speculative decoding configs assembled by hand; newer checkpoints that store the field under a different name.

Related errors

• num nextn_predict_layers is not in the config
• num nextn_predict_layers is not in the config
• online c128 does not support MTP
• Eagle3 MLA draft post_load_weights only supports float dtype
• Qwen3-Next MTP shared expert fusion currently supports exact

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3a2209c213c17f11. Report an issue: GitHub.