sgl-project/sglang · error · ValueError

Only 1 nextn layer is supported for Step3p5 checkpoints.

Error message

Only 1 nextn layer is supported for Step3p5 checkpoints.

What it means

Step3p5's MTP/nextn loading path assumes the checkpoint appends exactly one nextn (speculative draft) layer. If num_nextn_layers != 1 the loader refuses to proceed rather than mis-assigning layers.

Source

Thrown at python/sglang/srt/models/step3p5.py:946

            shard_id_matches = name_parts[4] == weight_parts[2]
            return shard_id_matches

        for name, loaded_weight in weights:
            # Filter nextn layer weights.
            if hasattr(self.config, "num_nextn_predict_layers"):
                num_nextn_layers = getattr(self.config, "num_nextn_predict_layers", 0)
                if num_nextn_layers and name.startswith("model.layers."):
                    layer_id = _get_layer_id_from_weight_name(name)
                    if layer_id is not None:
                        if not is_nextn:
                            # Normal load: skip layers appended after the main decoder.
                            if layer_id >= self.config.num_hidden_layers:
                                continue
                        else:
                            # nextn load: only keep the appended nextn layer.
                            # (Only 1 nextn layer is supported by current checkpoints.)
                            if num_nextn_layers != 1:
                                raise ValueError(
                                    "Only 1 nextn layer is supported for Step3p5 checkpoints."
                                )
                            nextn_layer_id = (
                                0
                                if self.config.num_hidden_layers == 1
                                else self.config.num_hidden_layers
                            )
                            if layer_id != nextn_layer_id:
                                # # nextn/MTP load: only keep the appended nextn layers.
                                # # Expected layer ids: [num_hidden_layers, num_hidden_layers + num_nextn_layers).
                                # start = self.config.num_hidden_layers
                                # end = self.config.num_hidden_layers + num_nextn_layers
                                # if not (start <= layer_id < end):
                                continue

            for param_name, weight_name, shard_id in stacked_params_mapping:
                if weight_name not in name:
                    continue

View on GitHub (pinned to 0132848349)

Solutions

1. Use a checkpoint with exactly one nextn layer
2. Set spec num_nextn_layers=1 (or disable speculative decoding to skip the nextn path)
3. Re-export the draft model containing only the first nextn layer

Example fix

# before
--speculative-algorithm EAGLE --speculative-num-steps 3 (multi-layer draft)
# after
use draft weights with num_nextn_layers = 1

n = draft_cfg.get('spec',{}).get('num_nextn_layers', 0)
if n: assert n == 1, f"num_nextn_layers={n}, only 1 supported"

When it happens

Trigger: Loading a Step3p5 checkpoint whose spec.num_nextn_layers is 0 or >1, or with num_hidden_layers==1 combined with multiple draft layers.

Common situations: Using a multi-layer MTP draft checkpoint with a runtime that supports only 1 draft layer; converting/merging checkpoints that add several nextn layers.

Related errors

• online c128 does not support MTP
• num_nextn_predict_layers is not in the config
• num nextn_predict_layers is not in the config
• num nextn_predict_layers is not in the config
• Qwen3-Next MTP shared expert fusion currently supports exact

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/582adb3d06ada66e. Report an issue: GitHub.