{"record":{"id":"1a0ec052f80539fb","repo":"sgl-project/sglang","slug":"standalone-speculative-decoding-requires-the-draft-1a0ec0","errorCode":null,"errorMessage":"STANDALONE speculative decoding requires the draft model to share the same vocabulary as the target model, but the two tokenizers have different token-to-id mappings even though their vocab sizes match. Use a draft model with a matching vocabulary, or a speculative algorithm that supports heterogeneous vocabularies.","messagePattern":"STANDALONE speculative decoding requires the draft model to share the same vocabulary as the target model, but the two tokenizers have different token-to-id mappings even though their vocab sizes match\\. Use a draft model with a matching vocabulary, or a speculative algorithm that supports heterogeneous vocabularies\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/speculative/standalone_worker_v2.py","lineNumber":226,"sourceCode":"        draft_vocab_size = self._draft_worker.draft_runner.model_config.vocab_size\n        draft_tokenizer = self._draft_worker.draft_worker.tokenizer\n        if target_vocab_size != draft_vocab_size:\n            raise ValueError(\n                f\"STANDALONE speculative decoding requires the draft model to share the \"\n                f\"same vocabulary as the target model, but got \"\n                f\"target vocab_size={target_vocab_size} and \"\n                f\"draft vocab_size={draft_vocab_size}. \"\n                f\"Use a draft model with a matching vocabulary, or a speculative \"\n                f\"algorithm that supports heterogeneous vocabularies.\"\n            )\n        if (\n            target_tokenizer is not None\n            and draft_tokenizer is not None\n            and hasattr(target_tokenizer, \"get_vocab\")\n            and hasattr(draft_tokenizer, \"get_vocab\")\n            and target_tokenizer.get_vocab() != draft_tokenizer.get_vocab()\n        ):\n            raise ValueError(\n                \"STANDALONE speculative decoding requires the draft model to share the \"\n                \"same vocabulary as the target model, but the two tokenizers have \"\n                \"different token-to-id mappings even though their vocab sizes match. \"\n                \"Use a draft model with a matching vocabulary, or a speculative \"\n                \"algorithm that supports heterogeneous vocabularies.\"\n            )\n","sourceCodeStart":208,"sourceCodeEnd":233,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/standalone_worker_v2.py#L208-L233","documentation":"Even when vocab sizes match numerically, STANDALONE requires identical token-to-id mappings; if both tokenizers expose get_vocab() and the dicts differ (different added tokens, different special tokens), verification would compare token ids from different vocabularies, so it raises ValueError.","triggerScenarios":"Draft and target tokenizers with equal vocab_size but different token->id maps, e.g. one has extra special tokens like <|im_start|> or different ordering of added tokens.","commonSituations":"Chat-tuned target with extra special tokens vs base draft model; custom tokenizers with modified added_tokens; tokenizer version drift between draft and target checkpoints.","solutions":["Use the target's tokenizer files for the draft model (copy tokenizer.json / tokenizer_config.json)","Regenerate/align added tokens so get_vocab() dicts match exactly","Pick a draft checkpoint released alongside the target model"],"exampleFix":"# before\ndraft: base tokenizer (no chat tokens), target: chat tokenizer -> raises\n# after\ncp target/tokenizer.json target/tokenizer_config.json draft_dir/  # then relaunch","handlingStrategy":"validation","validationCode":"tv = target_tokenizer.get_vocab(); dv = draft_tokenizer.get_vocab()\nassert tv == dv, 'token->id mappings differ; copy the target tokenizer into the draft dir'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Ship the target's tokenizer files with the draft model","Beware added special tokens silently changing mappings"],"tags":["speculative-decoding","tokenizer-mismatch","vocabulary-mismatch"],"backgroundTag":"tokenizer-vocab-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T11:17:15.048Z"}