{"record":{"id":"ceb5bac57036b185","repo":"vllm-project/vllm","slug":"use-heterogeneous-vocab-currently-only-supports-gr","errorCode":null,"errorMessage":"use_heterogeneous_vocab currently only supports greedy draft sampling. Set draft_sample_method='greedy' (the default) or omit it.","messagePattern":"use_heterogeneous_vocab currently only supports greedy draft sampling\\. Set draft_sample_method='greedy' \\(the default\\) or omit it\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1393,"sourceCode":"            or self.synthetic_acceptance_length is not None\n        ):\n            raise ValueError(\n                \"synthetic_acceptance_rates / synthetic_acceptance_length \"\n                \"are only valid with rejection_sample_method='synthetic'.\"\n            )\n\n        if self.draft_model_config:\n            self.draft_model_config.verify_with_parallel_config(\n                self.draft_parallel_config\n            )\n\n        if self.use_heterogeneous_vocab and not self.uses_draft_model():\n            raise ValueError(\n                \"use_heterogeneous_vocab only works with method='draft_model'\"\n            )\n\n        if self.use_heterogeneous_vocab and self.draft_sample_method != \"greedy\":\n            raise ValueError(\n                \"use_heterogeneous_vocab currently only supports greedy draft \"\n                \"sampling. Set draft_sample_method='greedy' (the default) or \"\n                \"omit it.\"\n            )\n\n        if not self.use_heterogeneous_vocab:\n            self.verify_equal_vocab_size_if_draft_model()\n        return self\n\n    def verify_equal_vocab_size_if_draft_model(self):\n        if (\n            self.method == \"draft_model\"\n            and self.target_model_config is not None\n            and self.draft_model_config is not None\n        ):\n            target_vocab_size = self.target_model_config.get_vocab_size()\n            draft_vocab_size = self.draft_model_config.get_vocab_size()\n            if target_vocab_size != draft_vocab_size:","sourceCodeStart":1375,"sourceCodeEnd":1411,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1375-L1411","documentation":"When use_heterogeneous_vocab=True, vLLM requires draft_sample_method='greedy'. Heterogeneous-vocab speculative decoding aligns draft and target distributions across different vocabularies, and that alignment is currently only implemented for greedy (argmax) draft sampling. Stochastic draft sampling would require translating probability distributions between vocabularies, which is not supported yet.","triggerScenarios":"Setting use_heterogeneous_vocab=True together with draft_sample_method set to anything other than 'greedy' (the default), e.g. draft_sample_method='sample' or a temperature-based sampler, in the speculative config.","commonSituations":"Enabling randomized draft sampling to improve acceptance diversity and then turning on heterogeneous vocab; carrying over draft_sample_method from a same-vocab setup where sampling worked; misreading the default and explicitly setting a non-greedy value.","solutions":["Remove draft_sample_method from the config (greedy is the default) so the check passes","Explicitly set draft_sample_method='greedy' alongside use_heterogeneous_vocab=True","If you need stochastic draft sampling, disable use_heterogeneous_vocab and use models with matching vocabularies"],"exampleFix":"# before\nspeculative_config = {\n    \"method\": \"draft_model\",\n    \"model\": \"draft\",\n    \"use_heterogeneous_vocab\": True,\n    \"draft_sample_method\": \"sample\",\n}\n# after\nspeculative_config = {\n    \"method\": \"draft_model\",\n    \"model\": \"draft\",\n    \"use_heterogeneous_vocab\": True,\n    \"draft_sample_method\": \"greedy\",\n}","handlingStrategy":"validation","validationCode":"spec = {\"method\": \"draft_model\", \"use_heterogeneous_vocab\": True,\n        \"draft_sample_method\": \"sample\"}\nif spec.get(\"use_heterogeneous_vocab\"):\n    assert spec.get(\"draft_sample_method\", \"greedy\") == \"greedy\", \\\n        \"heterogeneous vocab requires greedy draft sampling\"","typeGuard":"def valid_draft_sampling(spec: dict) -> bool:\n    return spec.get(\"draft_sample_method\", \"greedy\") == \"greedy\"","tryCatchPattern":null,"preventionTips":["Do not set draft_sample_method unless required; greedy is the default and always safe","Group heterogeneous-vocab flags together in one tested config snippet"],"tags":["speculative-decoding","config","sampling","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}