vllm-project/vllm · error · ValueError

The Inkling checkpoint does not contain MTP weights

Error message

The Inkling checkpoint does not contain MTP weights

What it means

When enabling MTP speculation on an Inkling checkpoint (model_type inkling_mm_model/inkling_model), vLLM reads mtp_config.num_nextn_predict_layers from the HF config; a value < 1 means the checkpoint ships no MTP head weights, so an MTP draft model cannot be built from it.

Source

Thrown at vllm/config/speculative.py:629

            n_predict = getattr(hf_config, "num_nextn_predict_layers", 1)
            hf_config.update({"n_predict": n_predict, "architectures": ["Step3p5MTP"]})

        if initial_architecture == "MistralLarge3ForCausalLM":
            hf_config.update({"architectures": ["EagleMistralLarge3ForCausalLM"]})

        if hf_config.model_type == "hy_v3":
            hf_config.model_type = "hy_v3_mtp"
            n_predict = getattr(hf_config, "num_nextn_predict_layers", None)
            hf_config.update(
                {"n_predict": n_predict, "architectures": ["HYV3MTPModel"]}
            )

        if hf_config.model_type in ("inkling_mm_model", "inkling_model"):
            mtp_config = getattr(hf_config, "mtp_config", None) or {}
            hf_config = getattr(hf_config, "text_config", hf_config)
            checkpoint_depths = mtp_config.get("num_nextn_predict_layers", 0)
            if checkpoint_depths < 1:
                raise ValueError("The Inkling checkpoint does not contain MTP weights")
            hf_config.model_type = "inkling_mtp"
            hf_config.update(
                {
                    "n_predict": checkpoint_depths,
                    "num_nextn_predict_layers": checkpoint_depths,
                    "chain_hidden_post_norm": mtp_config.get(
                        "chain_hidden_post_norm", False
                    ),
                    "local_layer_ids": mtp_config.get("local_layer_ids", []),
                    "architectures": ["InklingMTPModel"],
                }
            )

        if hf_config.model_type in ("gemma4_assistant", "gemma4_unified_assistant"):
            hf_config.model_type = "gemma4_mtp"
            text_config = getattr(hf_config, "text_config", hf_config)
            # The assistant runs all decoder layers in a single forward
            # call to produce one draft token, so n_predict=1.

View on GitHub (pinned to c794754062)

Solutions

1. Switch to the Inkling checkpoint variant that includes MTP weights (one whose mtp_config.num_nextn_predict_layers >= 1)
2. Or disable MTP speculation (drop --speculative-method mtp / num_speculative_tokens) for this base model
3. If you merged MTP weights yourself, ensure mtp_config in config.json declares the layer count

Example fix

# before
vllm serve inkling/inkling-base --speculative-method mtp --num-speculative-tokens 1

# after
vllm serve inkling/inkling-mtp --num-speculative-tokens 1

from transformers import AutoConfig

def inkling_has_mtp(model_id: str) -> bool:
    cfg = AutoConfig.from_pretrained(model_id)
    return (getattr(cfg, 'mtp_config', None) or {}).get('num_nextn_predict_layers', 0) >= 1

null

null

When it happens

Trigger: Running --speculative-method mtp against a base Inkling checkpoint released without the MTP layers; mtp_config missing or empty {} in config.json (defaults to 0); a distilled/pruned variant that drops MTP weights.

Common situations: Assuming every release of a family includes MTP layers; using the chat model instead of the reasoning/MTP variant; config.json fields renamed between releases.

Related errors

• mtp_layer_types must have one entry per MTP layer: got {len(
• target_model_config must be present for mtp
• rejection_sample_method='synthetic' requires exactly one of
• synthetic_acceptance_rates must have length {n}, got {rates}
• synthetic_acceptance_rates entries must be in [0, 1], got {r

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/7ad46c07073a2dc3. Report an issue: GitHub.