{"record":{"id":"99f0a919c2376bf6","repo":"sgl-project/sglang","slug":"standalone-speculative-decoding-requires-the-draft","errorCode":null,"errorMessage":"STANDALONE speculative decoding requires the draft model to share the same vocabulary as the target model, but got target vocab_size={target_vocab_size} and draft vocab_size={draft_vocab_size}. Use a draft model with a matching vocabulary, or a speculative algorithm that supports heterogeneous vocabularies.","messagePattern":"STANDALONE speculative decoding requires the draft model to share the same vocabulary as the target model, but got target vocab_size=(.+?) and draft vocab_size=(.+?)\\. Use a draft model with a matching vocabulary, or a speculative algorithm that supports heterogeneous vocabularies\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/speculative/standalone_worker_v2.py","lineNumber":211,"sourceCode":"            (), dtype=torch.int64, device=self.device\n        )\n        self.extend_lens = torch.empty((), dtype=torch.int64, device=self.device)\n\n        self.plan_stream, self.plan_stream_ctx = get_plan_stream(self.device)\n\n        # TODO: Adaptive speculative\n        self.adaptive_controller: Optional[AdaptiveController] = None\n\n    def _validate_vocab_compatibility(\n        self,\n        target_vocab_size: int,\n        target_tokenizer,\n    ) -> None:\n        \"\"\"Raise ValueError if the draft and target vocabularies are incompatible.\"\"\"\n        draft_vocab_size = self._draft_worker.draft_runner.model_config.vocab_size\n        draft_tokenizer = self._draft_worker.draft_worker.tokenizer\n        if target_vocab_size != draft_vocab_size:\n            raise ValueError(\n                f\"STANDALONE speculative decoding requires the draft model to share the \"\n                f\"same vocabulary as the target model, but got \"\n                f\"target vocab_size={target_vocab_size} and \"\n                f\"draft vocab_size={draft_vocab_size}. \"\n                f\"Use a draft model with a matching vocabulary, or a speculative \"\n                f\"algorithm that supports heterogeneous vocabularies.\"\n            )\n        if (\n            target_tokenizer is not None\n            and draft_tokenizer is not None\n            and hasattr(target_tokenizer, \"get_vocab\")\n            and hasattr(draft_tokenizer, \"get_vocab\")\n            and target_tokenizer.get_vocab() != draft_tokenizer.get_vocab()\n        ):\n            raise ValueError(\n                \"STANDALONE speculative decoding requires the draft model to share the \"\n                \"same vocabulary as the target model, but the two tokenizers have \"\n                \"different token-to-id mappings even though their vocab sizes match. \"","sourceCodeStart":193,"sourceCodeEnd":229,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/standalone_worker_v2.py#L193-L229","documentation":"STANDALONE speculative decoding (separate draft model) requires draft and target to share vocab size; _validate_vocab_compatibility raises ValueError when target vocab_size != draft vocab_size, since logits/argmax comparison across mismatched vocabularies is invalid.","triggerScenarios":"Launching with --speculative-algorithm STANDALONE (V2) where the draft model's config vocab_size differs from the target's, e.g. pairing an Llama-68M draft with a non-Llama target.","commonSituations":"Choosing a mismatched draft model (different tokenizer family); off-by-config wrong draft path; draft config with added special tokens changing vocab_size.","solutions":["Use a draft model with identical vocab size/tokenizer family as the target (e.g. same-family small model)","Verify both configs: compare target.config.vocab_size vs draft config vocab_size before launch","If vocabularies truly differ, switch to an algorithm supporting heterogeneous vocabularies (e.g. EAGLE-style with adapter) or drop spec decoding"],"exampleFix":"# before\n--speculative-algorithm STANDALONE --speculative-draft-model-path tiny_random bert\n# after\n--speculative-algorithm STANDALONE --speculative-draft-model-path <same-family small model>","handlingStrategy":"validation","validationCode":"from transformers import AutoConfig\nt = AutoConfig.from_pretrained(target_path).vocab_size\nd = AutoConfig.from_pretrained(draft_path).vocab_size\nassert t == d, f'vocab mismatch: {t} vs {d}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use same-family draft models released with the target","Check both configs' vocab_size before launching STANDALONE"],"tags":["speculative-decoding","vocabulary-mismatch","model-config"],"backgroundTag":"vocab-size-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}