vllm-project/vllm · error · ValueError

use_heterogeneous_vocab only works with method='draft_model'

Error message

use_heterogeneous_vocab only works with method='draft_model'

What it means

SpeculativeConfig raises this when use_heterogeneous_vocab=True but the speculative method is not 'draft_model'. Heterogeneous-vocabulary support (target and draft models with different tokenizer/vocab sizes) is only implemented for the draft-model method, where the draft model's logits can be projected onto the target vocabulary. With ngram, medusa, mtp, or eagle-style methods there is no separate draft vocabulary to reconcile.

Source

Thrown at vllm/config/speculative.py:1388

                self.synthetic_acceptance_length,
            )
            self.synthetic_acceptance_length = None
        elif (
            self.synthetic_acceptance_rates is not None
            or self.synthetic_acceptance_length is not None
        ):
            raise ValueError(
                "synthetic_acceptance_rates / synthetic_acceptance_length "
                "are only valid with rejection_sample_method='synthetic'."
            )

        if self.draft_model_config:
            self.draft_model_config.verify_with_parallel_config(
                self.draft_parallel_config
            )

        if self.use_heterogeneous_vocab and not self.uses_draft_model():
            raise ValueError(
                "use_heterogeneous_vocab only works with method='draft_model'"
            )

        if self.use_heterogeneous_vocab and self.draft_sample_method != "greedy":
            raise ValueError(
                "use_heterogeneous_vocab currently only supports greedy draft "
                "sampling. Set draft_sample_method='greedy' (the default) or "
                "omit it."
            )

        if not self.use_heterogeneous_vocab:
            self.verify_equal_vocab_size_if_draft_model()
        return self

    def verify_equal_vocab_size_if_draft_model(self):
        if (
            self.method == "draft_model"
            and self.target_model_config is not None

View on GitHub (pinned to c794754062)

Solutions

1. Set method='draft_model' and provide a real draft model (speculative_model=<draft>) so heterogeneous vocab handling is exercised
2. Remove use_heterogeneous_vocab (or set it to False) if you want to keep ngram/meduda/eagle/mtp — and ensure target/draft tokenizers actually match
3. If your target and draft genuinely have different vocab sizes, you must use the draft_model method; otherwise use models with identical tokenizers

Example fix

# before
speculative_config = {"method": "ngram", "use_heterogeneous_vocab": True}
# after
speculative_config = {
    "method": "draft_model",
    "model": "my-draft-model",
    "use_heterogeneous_vocab": True,
}

spec = {"method": "ngram", "use_heterogeneous_vocab": True}
assert not spec.get("use_heterogeneous_vocab") or spec.get("method") == "draft_model", \
    "use_heterogeneous_vocab requires method='draft_model'"

def supports_hetero_vocab(spec: dict) -> bool:
    return spec.get("method") == "draft_model"

When it happens

Trigger: Passing use_heterogeneous_vocab=True (or --speculative-config with that key) together with method='ngram', 'medusa', 'mtp', 'eagle', or 'synthetic' in the speculative config; the validator calls uses_draft_model() and it returns False.

Common situations: Trying to pair an ngram/eagle draft setup with models that have different tokenizers (e.g. different tokenizer versions of the same family); enabling the flag experimentally after seeing it in docs without realizing it is draft-model-only; migrating a config between speculative methods and leaving the flag set.

Related errors

• Target and draft model should have the same vocabulary size.
• ReasoningConfig: failed to tokenize reasoning strings: reaso
• rejection_sample_method='synthetic' requires exactly one of
• synthetic_acceptance_rates must have length {n}, got {rates}
• synthetic_acceptance_rates entries must be in [0, 1], got {r

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1a38c3e3edf021d6. Report an issue: GitHub.