vllm-project/vllm · error · ValueError

mtp_layer_types must have one entry per MTP layer: got {len(

Error message

mtp_layer_types must have one entry per MTP layer: got {len(mtp_layer_types)} for {n_predict} layers

What it means

For dots3_note checkpoints, hf_config_override appends one layer_types entry per MTP (multi-token prediction) layer. If the checkpoint's mtp_layer_types list length differs from num_nextn_predict_layers, the per-layer type map would be misaligned with the actual decoder layers, so the config override is rejected.

Source

Thrown at vllm/config/speculative.py:350

                None,
            )
            if layer_ids is not None:
                # Convert to tuple to make it hashable
                factors.append(tuple(layer_ids))

        hash_str = safe_hash(str(factors).encode(), usedforsecurity=False).hexdigest()
        return hash_str

    @staticmethod
    def hf_config_override(hf_config: PretrainedConfig) -> PretrainedConfig:
        initial_architecture = hf_config.architectures[0]
        if hf_config.model_type == "dots3_note":
            n_predict = getattr(hf_config, "num_nextn_predict_layers", 1)
            mtp_layer_types = getattr(hf_config, "mtp_layer_types", None)
            if mtp_layer_types is None:
                mtp_layer_types = ["sliding_attention"] * n_predict
            if len(mtp_layer_types) != n_predict:
                raise ValueError(
                    "mtp_layer_types must have one entry per MTP layer: "
                    f"got {len(mtp_layer_types)} for {n_predict} layers"
                )
            hf_config.layer_types = [*hf_config.layer_types, *mtp_layer_types]
            hf_config.model_type = "dots3_note_mtp"
            hf_config.update(
                {"n_predict": n_predict, "architectures": ["Dots3NoteMTPModel"]}
            )
        if hf_config.model_type in (
            "deepseek_v3",
            "deepseek_v32",
            "glm_moe_dsa",
        ):
            hf_config.model_type = "deepseek_mtp"
        if hf_config.model_type == "deepseek_mtp":
            n_predict = getattr(hf_config, "num_nextn_predict_layers", None)
            hf_config.update(
                {"n_predict": n_predict, "architectures": ["DeepSeekMTPModel"]}

View on GitHub (pinned to c794754062)

Solutions

1. Edit config.json so len(mtp_layer_types) == num_nextn_predict_layers
2. Or delete mtp_layer_types from config.json; the code then defaults every MTP layer to 'sliding_attention'
3. Re-download/verify the checkpoint if you did not modify it

Example fix

// config.json before
"num_nextn_predict_layers": 3, "mtp_layer_types": ["sliding_attention"]

// config.json after
"num_nextn_predict_layers": 3, "mtp_layer_types": ["sliding_attention", "sliding_attention", "sliding_attention"]

def mtp_types_consistent(cfg) -> bool:
    n = getattr(cfg, 'num_nextn_predict_layers', 1)
    t = getattr(cfg, 'mtp_layer_types', None)
    return t is None or len(t) == n

null

null

When it happens

Trigger: Loading a dots3_note checkpoint whose config.json has mtp_layer_types with 2 entries but num_nextn_predict_layers=3 (or vice versa); hand-edited config.json adding one field but not the other; upstream checkpoint release with inconsistent fields.

Common situations: Editing num_nextn_predict_layers to change MTP depth without resizing mtp_layer_types; merging configs across checkpoint revisions.

Related errors

• The Inkling checkpoint does not contain MTP weights
• target_model_config must be present for mtp
• rejection_sample_method='synthetic' requires exactly one of
• synthetic_acceptance_rates must have length {n}, got {rates}
• synthetic_acceptance_rates entries must be in [0, 1], got {r

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/79a7569ef4d3db38. Report an issue: GitHub.