{"record":{"id":"00d1a65c1c946b44","repo":"vllm-project/vllm","slug":"target-model-config-must-be-present-for-mtp","errorCode":null,"errorMessage":"target_model_config must be present for mtp","messagePattern":"target_model_config must be present for mtp","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":761,"sourceCode":"            self.model\n        ):\n            self.method = \"custom_class\"\n        elif self.method is None:\n            if self.model in (\"ngram\", \"[ngram]\"):\n                self.method = \"ngram\"\n            else:\n                self.method = \"draft_model\"\n\n        if self.method in get_args(MTPModelTypes) and self.method != \"mtp\":\n            logger.warning(\n                \"method `%s` is deprecated and replaced with mtp.\", self.method\n            )\n            self.method = \"mtp\"\n\n        if self.model is None and self.num_speculative_tokens is not None:\n            if self.method == \"mtp\":\n                if self.target_model_config is None:\n                    raise ValueError(\"target_model_config must be present for mtp\")\n                if self.target_model_config.hf_text_config.model_type == \"deepseek_v32\":\n                    # FIXME(luccafong): cudagraph with v32 MTP is not supported,\n                    # remove this when the issue is fixed.\n                    self.enforce_eager = True\n                # use the draft model from the same model:\n                self.model = self.target_model_config.model\n                # Align the quantization of draft model for cases such as\n                # --quantization fp8 with a bf16 checkpoint.\n                if not self.quantization:\n                    self.quantization = self.target_model_config.quantization\n            elif self.method == \"dspark\":\n                # DeepSeek DSpark can ship the weights inside the target checkpoint\n                if self.target_model_config is None:\n                    raise ValueError(\"target_model_config must be present for dspark\")\n                self.model = self.target_model_config.model\n                if not self.quantization:\n                    self.quantization = self.target_model_config.quantization\n            elif self.method in (\"ngram\", \"[ngram]\"):","sourceCodeStart":743,"sourceCodeEnd":779,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L743-L779","documentation":"Inside SpeculativeConfig init: when no draft model is given but num_speculative_tokens is set and method == 'mtp', the MTP head is taken from the target model itself, which requires the target ModelConfig. If target_model_config is None the MTP path cannot resolve the draft weights or align quantization, so it fails fast.","triggerScenarios":"Constructing SpeculativeConfig(method='mtp', num_speculative_tokens=N) directly without target_model_config (normally injected by vLLM from the engine's model config); custom engine wrappers/ports that build SpeculativeConfig standalone; internal refactor changing when target_model_config is attached.","commonSituations":"Programmatic use of SpeculativeConfig outside the standard LLM/AsyncLLM entrypoints; middleware that re-validates configs before the engine attaches the target.","solutions":["Use the standard entrypoints (LLM/AsyncLLMEngine/vllm serve) so vLLM supplies target_model_config","If constructing manually, pass target_model_config=ModelConfig(the target model) alongside method='mtp'","Alternatively provide an explicit draft via model= instead of relying on same-checkpoint MTP"],"exampleFix":"# before\nSpeculativeConfig(method='mtp', num_speculative_tokens=1)\n\n# after\nSpeculativeConfig(method='mtp', num_speculative_tokens=1, target_model_config=ModelConfig(model='deepseek-ai/DeepSeek-V3'))","handlingStrategy":"validation","validationCode":"def mtp_config_ready(cfg: 'SpeculativeConfig') -> bool:\n    return not (cfg.method == 'mtp' and cfg.model is None and cfg.num_speculative_tokens is not None and cfg.target_model_config is None)","typeGuard":"null","tryCatchPattern":"null","preventionTips":["Build SpeculativeConfig through LLM/AsyncLLM so target_model_config is injected","Never construct method='mtp' configs standalone without the target ModelConfig"],"tags":["speculative-decoding","mtp","config","api-misuse"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}