{"record":{"id":"7ad46c07073a2dc3","repo":"vllm-project/vllm","slug":"the-inkling-checkpoint-does-not-contain-mtp-weight","errorCode":null,"errorMessage":"The Inkling checkpoint does not contain MTP weights","messagePattern":"The Inkling checkpoint does not contain MTP weights","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":629,"sourceCode":"            n_predict = getattr(hf_config, \"num_nextn_predict_layers\", 1)\n            hf_config.update({\"n_predict\": n_predict, \"architectures\": [\"Step3p5MTP\"]})\n\n        if initial_architecture == \"MistralLarge3ForCausalLM\":\n            hf_config.update({\"architectures\": [\"EagleMistralLarge3ForCausalLM\"]})\n\n        if hf_config.model_type == \"hy_v3\":\n            hf_config.model_type = \"hy_v3_mtp\"\n            n_predict = getattr(hf_config, \"num_nextn_predict_layers\", None)\n            hf_config.update(\n                {\"n_predict\": n_predict, \"architectures\": [\"HYV3MTPModel\"]}\n            )\n\n        if hf_config.model_type in (\"inkling_mm_model\", \"inkling_model\"):\n            mtp_config = getattr(hf_config, \"mtp_config\", None) or {}\n            hf_config = getattr(hf_config, \"text_config\", hf_config)\n            checkpoint_depths = mtp_config.get(\"num_nextn_predict_layers\", 0)\n            if checkpoint_depths < 1:\n                raise ValueError(\"The Inkling checkpoint does not contain MTP weights\")\n            hf_config.model_type = \"inkling_mtp\"\n            hf_config.update(\n                {\n                    \"n_predict\": checkpoint_depths,\n                    \"num_nextn_predict_layers\": checkpoint_depths,\n                    \"chain_hidden_post_norm\": mtp_config.get(\n                        \"chain_hidden_post_norm\", False\n                    ),\n                    \"local_layer_ids\": mtp_config.get(\"local_layer_ids\", []),\n                    \"architectures\": [\"InklingMTPModel\"],\n                }\n            )\n\n        if hf_config.model_type in (\"gemma4_assistant\", \"gemma4_unified_assistant\"):\n            hf_config.model_type = \"gemma4_mtp\"\n            text_config = getattr(hf_config, \"text_config\", hf_config)\n            # The assistant runs all decoder layers in a single forward\n            # call to produce one draft token, so n_predict=1.","sourceCodeStart":611,"sourceCodeEnd":647,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L611-L647","documentation":"When enabling MTP speculation on an Inkling checkpoint (model_type inkling_mm_model/inkling_model), vLLM reads mtp_config.num_nextn_predict_layers from the HF config; a value < 1 means the checkpoint ships no MTP head weights, so an MTP draft model cannot be built from it.","triggerScenarios":"Running --speculative-method mtp against a base Inkling checkpoint released without the MTP layers; mtp_config missing or empty {} in config.json (defaults to 0); a distilled/pruned variant that drops MTP weights.","commonSituations":"Assuming every release of a family includes MTP layers; using the chat model instead of the reasoning/MTP variant; config.json fields renamed between releases.","solutions":["Switch to the Inkling checkpoint variant that includes MTP weights (one whose mtp_config.num_nextn_predict_layers >= 1)","Or disable MTP speculation (drop --speculative-method mtp / num_speculative_tokens) for this base model","If you merged MTP weights yourself, ensure mtp_config in config.json declares the layer count"],"exampleFix":"# before\nvllm serve inkling/inkling-base --speculative-method mtp --num-speculative-tokens 1\n\n# after\nvllm serve inkling/inkling-mtp --num-speculative-tokens 1","handlingStrategy":"validation","validationCode":"from transformers import AutoConfig\n\ndef inkling_has_mtp(model_id: str) -> bool:\n    cfg = AutoConfig.from_pretrained(model_id)\n    return (getattr(cfg, 'mtp_config', None) or {}).get('num_nextn_predict_layers', 0) >= 1","typeGuard":"null","tryCatchPattern":"null","preventionTips":["Choose the MTP-bearing checkpoint variant before enabling --speculative-method mtp","Check mtp_config in config.json when preparing Inkling deployments"],"tags":["speculative-decoding","mtp","checkpoint","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}