sgl-project/sglang · critical · ValueError

STANDALONE speculative decoding requires the draft model to

Error message

STANDALONE speculative decoding requires the draft model to share the same vocabulary as the target model, but the two tokenizers have different token-to-id mappings even though their vocab sizes match. Use a draft model with a matching vocabulary, or a speculative algorithm that supports heterogeneous vocabularies.

What it means

Even when vocab sizes match numerically, STANDALONE requires identical token-to-id mappings; if both tokenizers expose get_vocab() and the dicts differ (different added tokens, different special tokens), verification would compare token ids from different vocabularies, so it raises ValueError.

Source

Thrown at python/sglang/srt/speculative/standalone_worker_v2.py:226

        draft_vocab_size = self._draft_worker.draft_runner.model_config.vocab_size
        draft_tokenizer = self._draft_worker.draft_worker.tokenizer
        if target_vocab_size != draft_vocab_size:
            raise ValueError(
                f"STANDALONE speculative decoding requires the draft model to share the "
                f"same vocabulary as the target model, but got "
                f"target vocab_size={target_vocab_size} and "
                f"draft vocab_size={draft_vocab_size}. "
                f"Use a draft model with a matching vocabulary, or a speculative "
                f"algorithm that supports heterogeneous vocabularies."
            )
        if (
            target_tokenizer is not None
            and draft_tokenizer is not None
            and hasattr(target_tokenizer, "get_vocab")
            and hasattr(draft_tokenizer, "get_vocab")
            and target_tokenizer.get_vocab() != draft_tokenizer.get_vocab()
        ):
            raise ValueError(
                "STANDALONE speculative decoding requires the draft model to share the "
                "same vocabulary as the target model, but the two tokenizers have "
                "different token-to-id mappings even though their vocab sizes match. "
                "Use a draft model with a matching vocabulary, or a speculative "
                "algorithm that supports heterogeneous vocabularies."
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Use the target's tokenizer files for the draft model (copy tokenizer.json / tokenizer_config.json)
2. Regenerate/align added tokens so get_vocab() dicts match exactly
3. Pick a draft checkpoint released alongside the target model

Example fix

# before
draft: base tokenizer (no chat tokens), target: chat tokenizer -> raises
# after
cp target/tokenizer.json target/tokenizer_config.json draft_dir/  # then relaunch

tv = target_tokenizer.get_vocab(); dv = draft_tokenizer.get_vocab()
assert tv == dv, 'token->id mappings differ; copy the target tokenizer into the draft dir'

When it happens

Trigger: Draft and target tokenizers with equal vocab_size but different token->id maps, e.g. one has extra special tokens like <|im_start|> or different ordering of added tokens.

Common situations: Chat-tuned target with extra special tokens vs base draft model; custom tokenizers with modified added_tokens; tokenizer version drift between draft and target checkpoints.

Related errors

• STANDALONE speculative decoding requires the draft model to
• This browser cannot encode H.264 MP4
• kv-canary: launch_canary_plan_kernels_torch_reference verify
• HiSparse speculative swap requires 2-4 steps, got {num_steps
• Invalid stacked fused KV projection shape: got {tuple(kv.sha

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1a0ec052f80539fb. Report an issue: GitHub.