{"record":{"id":"028f0ed0c513ef6e","repo":"sgl-project/sglang","slug":"dflash-mask-token-id-is-outside-the-target-vocab-s","errorCode":null,"errorMessage":"DFLASH mask_token_id is outside the target vocab size. mask_token_id={resolved_id}, vocab_size={vocab_size}. This likely means mask_token={mask_token!r} requires vocab expansion beyond the model's embedding size. SGLang does not support resizing target embeddings for DFLASH yet.","messagePattern":"DFLASH mask_token_id is outside the target vocab size\\. mask_token_id=(.+?), vocab_size=(.+?)\\. This likely means mask_token=(.+?) requires vocab expansion beyond the model's embedding size\\. SGLang does not support resizing target embeddings for DFLASH yet\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/speculative/dflash_worker_v2.py","lineNumber":884,"sourceCode":"                draft_prefix_lens,\n                block_end,\n                verify_out_cache_loc_2d.reshape(-1),\n                bs,\n            )\n\n    def _resolve_mask_token_id(\n        self, *, mask_token: str, mask_token_id: Optional[int] = None\n    ) -> int:\n        if not isinstance(mask_token, str) or not mask_token:\n            raise ValueError(\n                f\"DFLASH mask_token must be a non-empty string, got {mask_token!r}.\"\n            )\n\n        vocab_size = int(self.target_worker.model_runner.model_config.vocab_size)\n        if mask_token_id is not None:\n            resolved_id = int(mask_token_id)\n            if resolved_id >= vocab_size:\n                raise ValueError(\n                    \"DFLASH mask_token_id is outside the target vocab size. \"\n                    f\"mask_token_id={resolved_id}, vocab_size={vocab_size}. \"\n                    f\"This likely means mask_token={mask_token!r} requires vocab expansion beyond the model's embedding size. \"\n                    \"SGLang does not support resizing target embeddings for DFLASH yet.\"\n                )\n\n            tokenizer = getattr(self.target_worker, \"tokenizer\", None)\n            if tokenizer is not None:\n                token_id_from_vocab = tokenizer.get_vocab().get(mask_token, None)\n                if (\n                    token_id_from_vocab is not None\n                    and int(token_id_from_vocab) != resolved_id\n                ):\n                    raise ValueError(\n                        \"DFLASH config mismatch: dflash_config.mask_token_id conflicts with tokenizer vocab id \"\n                        f\"for dflash_config.mask_token. mask_token={mask_token!r}, \"\n                        f\"mask_token_id={resolved_id}, tokenizer_vocab_id={int(token_id_from_vocab)}.\"\n                    )","sourceCodeStart":866,"sourceCodeEnd":902,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/dflash_worker_v2.py#L866-L902","documentation":"Raised during DFLASH worker init when the resolved mask token id is >= the target model's vocab size. DFLASH drafts like Nemotron 3.5 add a mask token whose id may lie beyond the original vocab; sglang cannot resize the target model's embedding matrix to accommodate it, so it refuses to start. The message explicitly calls out that embedding expansion for DFLASH is unsupported.","triggerScenarios":"Loading a DFLASH draft config whose mask_token_id (or tokenizer-resolved mask token) falls outside target vocab_size — typically a draft checkpoint with an extended vocab (extra special tokens) paired with a target model whose embeddings were never expanded.","commonSituations":"Using a tokenizer/draft checkpoint revision that added the mask token past the vocab boundary while the target model weights keep the original vocab; mixing model components from incompatible revisions; explicitly overriding mask_token_id with a value beyond vocab_size.","solutions":["Use matching target + draft + tokenizer revisions released together (e.g. the official Nemotron 3.5 bundle) so the mask token is within the target vocab","Check tokenizer.convert_tokens_to_ids(mask_token) vs model_config.vocab_size; if the id is out of range, your tokenizer/target mismatch is the root cause","If you control the checkpoints, remap or retrain so the mask token id sits inside the target vocab — sglang cannot resize target embeddings for DFLASH today","Do not hand-override mask_token_id to an out-of-vocab value; remove the override and let resolution use the tokenizer"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"from transformers import AutoTokenizer\ntok = AutoTokenizer.from_pretrained(draft_path)\nmask_id = tok.convert_tokens_to_ids(mask_token)\nvocab = target_model_config.vocab_size\nassert mask_id is not None and mask_id < vocab, (\n    f\"mask token id {mask_id} outside target vocab {vocab}; \"\n    \"use matching target/draft/tokenizer revisions\"\n)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always load target model, draft model, and tokenizer from the same official release bundle","Check mask token id against vocab_size before launching a DFLASH server","Never hand-set mask_token_id beyond the target embedding size"],"tags":["sglang","speculative-decoding","dflash","vocab-size","embedding","model-loading"],"backgroundTag":"token-id-out-of-vocab","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}