vllm-project/vllm · error · ValueError

target_model_config must be present for mtp

Error message

target_model_config must be present for mtp

What it means

Inside SpeculativeConfig init: when no draft model is given but num_speculative_tokens is set and method == 'mtp', the MTP head is taken from the target model itself, which requires the target ModelConfig. If target_model_config is None the MTP path cannot resolve the draft weights or align quantization, so it fails fast.

Source

Thrown at vllm/config/speculative.py:761

            self.model
        ):
            self.method = "custom_class"
        elif self.method is None:
            if self.model in ("ngram", "[ngram]"):
                self.method = "ngram"
            else:
                self.method = "draft_model"

        if self.method in get_args(MTPModelTypes) and self.method != "mtp":
            logger.warning(
                "method `%s` is deprecated and replaced with mtp.", self.method
            )
            self.method = "mtp"

        if self.model is None and self.num_speculative_tokens is not None:
            if self.method == "mtp":
                if self.target_model_config is None:
                    raise ValueError("target_model_config must be present for mtp")
                if self.target_model_config.hf_text_config.model_type == "deepseek_v32":
                    # FIXME(luccafong): cudagraph with v32 MTP is not supported,
                    # remove this when the issue is fixed.
                    self.enforce_eager = True
                # use the draft model from the same model:
                self.model = self.target_model_config.model
                # Align the quantization of draft model for cases such as
                # --quantization fp8 with a bf16 checkpoint.
                if not self.quantization:
                    self.quantization = self.target_model_config.quantization
            elif self.method == "dspark":
                # DeepSeek DSpark can ship the weights inside the target checkpoint
                if self.target_model_config is None:
                    raise ValueError("target_model_config must be present for dspark")
                self.model = self.target_model_config.model
                if not self.quantization:
                    self.quantization = self.target_model_config.quantization
            elif self.method in ("ngram", "[ngram]"):

View on GitHub (pinned to c794754062)

Solutions

1. Use the standard entrypoints (LLM/AsyncLLMEngine/vllm serve) so vLLM supplies target_model_config
2. If constructing manually, pass target_model_config=ModelConfig(the target model) alongside method='mtp'
3. Alternatively provide an explicit draft via model= instead of relying on same-checkpoint MTP

Example fix

# before
SpeculativeConfig(method='mtp', num_speculative_tokens=1)

# after
SpeculativeConfig(method='mtp', num_speculative_tokens=1, target_model_config=ModelConfig(model='deepseek-ai/DeepSeek-V3'))

def mtp_config_ready(cfg: 'SpeculativeConfig') -> bool:
    return not (cfg.method == 'mtp' and cfg.model is None and cfg.num_speculative_tokens is not None and cfg.target_model_config is None)

null

null

When it happens

Trigger: Constructing SpeculativeConfig(method='mtp', num_speculative_tokens=N) directly without target_model_config (normally injected by vLLM from the engine's model config); custom engine wrappers/ports that build SpeculativeConfig standalone; internal refactor changing when target_model_config is attached.

Common situations: Programmatic use of SpeculativeConfig outside the standard LLM/AsyncLLM entrypoints; middleware that re-validates configs before the engine attaches the target.

Related errors

• mtp_layer_types must have one entry per MTP layer: got {len(
• The Inkling checkpoint does not contain MTP weights
• target_model_config must be present for dspark
• method='custom_class' requires 'model' to contain the custom
• num_speculative_tokens was provided but without speculative

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/00d1a65c1c946b44. Report an issue: GitHub.