{"record":{"id":"1a38c3e3edf021d6","repo":"vllm-project/vllm","slug":"use-heterogeneous-vocab-only-works-with-method-dr","errorCode":null,"errorMessage":"use_heterogeneous_vocab only works with method='draft_model'","messagePattern":"use_heterogeneous_vocab only works with method='draft_model'","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1388,"sourceCode":"                self.synthetic_acceptance_length,\n            )\n            self.synthetic_acceptance_length = None\n        elif (\n            self.synthetic_acceptance_rates is not None\n            or self.synthetic_acceptance_length is not None\n        ):\n            raise ValueError(\n                \"synthetic_acceptance_rates / synthetic_acceptance_length \"\n                \"are only valid with rejection_sample_method='synthetic'.\"\n            )\n\n        if self.draft_model_config:\n            self.draft_model_config.verify_with_parallel_config(\n                self.draft_parallel_config\n            )\n\n        if self.use_heterogeneous_vocab and not self.uses_draft_model():\n            raise ValueError(\n                \"use_heterogeneous_vocab only works with method='draft_model'\"\n            )\n\n        if self.use_heterogeneous_vocab and self.draft_sample_method != \"greedy\":\n            raise ValueError(\n                \"use_heterogeneous_vocab currently only supports greedy draft \"\n                \"sampling. Set draft_sample_method='greedy' (the default) or \"\n                \"omit it.\"\n            )\n\n        if not self.use_heterogeneous_vocab:\n            self.verify_equal_vocab_size_if_draft_model()\n        return self\n\n    def verify_equal_vocab_size_if_draft_model(self):\n        if (\n            self.method == \"draft_model\"\n            and self.target_model_config is not None","sourceCodeStart":1370,"sourceCodeEnd":1406,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1370-L1406","documentation":"SpeculativeConfig raises this when use_heterogeneous_vocab=True but the speculative method is not 'draft_model'. Heterogeneous-vocabulary support (target and draft models with different tokenizer/vocab sizes) is only implemented for the draft-model method, where the draft model's logits can be projected onto the target vocabulary. With ngram, medusa, mtp, or eagle-style methods there is no separate draft vocabulary to reconcile.","triggerScenarios":"Passing use_heterogeneous_vocab=True (or --speculative-config with that key) together with method='ngram', 'medusa', 'mtp', 'eagle', or 'synthetic' in the speculative config; the validator calls uses_draft_model() and it returns False.","commonSituations":"Trying to pair an ngram/eagle draft setup with models that have different tokenizers (e.g. different tokenizer versions of the same family); enabling the flag experimentally after seeing it in docs without realizing it is draft-model-only; migrating a config between speculative methods and leaving the flag set.","solutions":["Set method='draft_model' and provide a real draft model (speculative_model=<draft>) so heterogeneous vocab handling is exercised","Remove use_heterogeneous_vocab (or set it to False) if you want to keep ngram/meduda/eagle/mtp — and ensure target/draft tokenizers actually match","If your target and draft genuinely have different vocab sizes, you must use the draft_model method; otherwise use models with identical tokenizers"],"exampleFix":"# before\nspeculative_config = {\"method\": \"ngram\", \"use_heterogeneous_vocab\": True}\n# after\nspeculative_config = {\n    \"method\": \"draft_model\",\n    \"model\": \"my-draft-model\",\n    \"use_heterogeneous_vocab\": True,\n}","handlingStrategy":"validation","validationCode":"spec = {\"method\": \"ngram\", \"use_heterogeneous_vocab\": True}\nassert not spec.get(\"use_heterogeneous_vocab\") or spec.get(\"method\") == \"draft_model\", \\\n    \"use_heterogeneous_vocab requires method='draft_model'\"","typeGuard":"def supports_hetero_vocab(spec: dict) -> bool:\n    return spec.get(\"method\") == \"draft_model\"","tryCatchPattern":null,"preventionTips":["Treat use_heterogeneous_vocab as a draft_model-method-exclusive flag in config templates","Lint speculative configs for cross-method flag leakage in CI"],"tags":["speculative-decoding","config","tokenizer","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}