{"record":{"id":"79a7569ef4d3db38","repo":"vllm-project/vllm","slug":"mtp-layer-types-must-have-one-entry-per-mtp-layer","errorCode":null,"errorMessage":"mtp_layer_types must have one entry per MTP layer: got {len(mtp_layer_types)} for {n_predict} layers","messagePattern":"mtp_layer_types must have one entry per MTP layer: got (.+?) for (.+?) layers","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":350,"sourceCode":"                None,\n            )\n            if layer_ids is not None:\n                # Convert to tuple to make it hashable\n                factors.append(tuple(layer_ids))\n\n        hash_str = safe_hash(str(factors).encode(), usedforsecurity=False).hexdigest()\n        return hash_str\n\n    @staticmethod\n    def hf_config_override(hf_config: PretrainedConfig) -> PretrainedConfig:\n        initial_architecture = hf_config.architectures[0]\n        if hf_config.model_type == \"dots3_note\":\n            n_predict = getattr(hf_config, \"num_nextn_predict_layers\", 1)\n            mtp_layer_types = getattr(hf_config, \"mtp_layer_types\", None)\n            if mtp_layer_types is None:\n                mtp_layer_types = [\"sliding_attention\"] * n_predict\n            if len(mtp_layer_types) != n_predict:\n                raise ValueError(\n                    \"mtp_layer_types must have one entry per MTP layer: \"\n                    f\"got {len(mtp_layer_types)} for {n_predict} layers\"\n                )\n            hf_config.layer_types = [*hf_config.layer_types, *mtp_layer_types]\n            hf_config.model_type = \"dots3_note_mtp\"\n            hf_config.update(\n                {\"n_predict\": n_predict, \"architectures\": [\"Dots3NoteMTPModel\"]}\n            )\n        if hf_config.model_type in (\n            \"deepseek_v3\",\n            \"deepseek_v32\",\n            \"glm_moe_dsa\",\n        ):\n            hf_config.model_type = \"deepseek_mtp\"\n        if hf_config.model_type == \"deepseek_mtp\":\n            n_predict = getattr(hf_config, \"num_nextn_predict_layers\", None)\n            hf_config.update(\n                {\"n_predict\": n_predict, \"architectures\": [\"DeepSeekMTPModel\"]}","sourceCodeStart":332,"sourceCodeEnd":368,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L332-L368","documentation":"For dots3_note checkpoints, hf_config_override appends one layer_types entry per MTP (multi-token prediction) layer. If the checkpoint's mtp_layer_types list length differs from num_nextn_predict_layers, the per-layer type map would be misaligned with the actual decoder layers, so the config override is rejected.","triggerScenarios":"Loading a dots3_note checkpoint whose config.json has mtp_layer_types with 2 entries but num_nextn_predict_layers=3 (or vice versa); hand-edited config.json adding one field but not the other; upstream checkpoint release with inconsistent fields.","commonSituations":"Editing num_nextn_predict_layers to change MTP depth without resizing mtp_layer_types; merging configs across checkpoint revisions.","solutions":["Edit config.json so len(mtp_layer_types) == num_nextn_predict_layers","Or delete mtp_layer_types from config.json; the code then defaults every MTP layer to 'sliding_attention'","Re-download/verify the checkpoint if you did not modify it"],"exampleFix":"// config.json before\n\"num_nextn_predict_layers\": 3, \"mtp_layer_types\": [\"sliding_attention\"]\n\n// config.json after\n\"num_nextn_predict_layers\": 3, \"mtp_layer_types\": [\"sliding_attention\", \"sliding_attention\", \"sliding_attention\"]","handlingStrategy":"validation","validationCode":"def mtp_types_consistent(cfg) -> bool:\n    n = getattr(cfg, 'num_nextn_predict_layers', 1)\n    t = getattr(cfg, 'mtp_layer_types', None)\n    return t is None or len(t) == n","typeGuard":"null","tryCatchPattern":"null","preventionTips":["Edit num_nextn_predict_layers and mtp_layer_types together in config.json","Validate checkpoint config.json before serving in CI"],"tags":["speculative-decoding","mtp","checkpoint","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}