{"record":{"id":"95678f67e8c84142","repo":"sgl-project/sglang","slug":"nemotron-3-5-dflash-draft-requires-its-checkpoint","errorCode":null,"errorMessage":"Nemotron 3.5 DFLASH draft requires its checkpoint embedding.","messagePattern":"Nemotron 3\\.5 DFLASH draft requires its checkpoint embedding\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/speculative/dflash_worker_v2.py","lineNumber":185,"sourceCode":"        torch.gather(gathered_ids.view(self.tp_size, n), 0, best_rank, out=selected)\n        self.out[:n].copy_(selected.view(-1))\n\n\ndef _commit_accept(candidates, accept_len, bonus_tokens):\n    \"\"\"The committed block: drafted tokens shifted left, the bonus at the accept\n    boundary. Returns it with the commit lengths.\"\"\"\n    out_tokens = torch.empty_like(candidates, dtype=torch.int64)\n    out_tokens[:, :-1].copy_(candidates[:, 1:])\n    out_tokens[:, -1].fill_(0)\n    out_tokens.scatter_(1, accept_len.to(torch.int64)[:, None], bonus_tokens[:, None])\n    return out_tokens, accept_len.to(torch.int32) + 1\n\n\ndef _resolve_dflash_embedding_module(draft_model, target_model):\n    if getattr(draft_model, \"is_nemotron_35_draft\", False):\n        embed_module = draft_model.get_input_embeddings()\n        if embed_module is None:\n            raise RuntimeError(\n                \"Nemotron 3.5 DFLASH draft requires its checkpoint embedding.\"\n            )\n        return embed_module\n    return target_model.get_input_embeddings()\n\n\ndef _is_all_greedy(sampling_info) -> bool:\n    return sampling_info is None or sampling_info.is_all_greedy\n\n\ndef _selector_lattice(draft_model, pred_hidden, anchor_token_ids):\n    # Flattened to [N, H] and viewed back because the radix top-k kernel is 2D.\n    bs, num_pred = pred_hidden.shape[0], pred_hidden.shape[1]\n    candidate_ids, unary_logits = draft_model.compute_candidates(\n        pred_hidden.reshape(-1, pred_hidden.shape[-1])\n    )\n    candidate_ids = candidate_ids.view(bs, num_pred, -1)\n    return candidate_ids, draft_model.candidate_selector.build_lattice(","sourceCodeStart":167,"sourceCodeEnd":203,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/dflash_worker_v2.py#L167-L203","documentation":"Thrown by the DFLASH v2 worker when the draft model is flagged as a Nemotron 3.5 draft (is_nemotron_35_draft) but get_input_embeddings() returns None. Nemotron 3.5 DFLASH drafts must use their own checkpoint's embedding matrix instead of the target model's, so a missing embedding makes the worker unable to embed draft inputs. It usually indicates a broken/partially-loaded draft checkpoint or a model class that doesn't implement the embedding accessor correctly.","triggerScenarios":"Loading a Nemotron 3.5 DFLASH draft checkpoint whose weights lack the embedding (e.g. sharded file missing, or tie_word_embeddings handling stripped it); a custom draft model class overriding get_input_embeddings to return None; quantized/converted checkpoints that drop embed_tokens.","commonSituations":"Swapping in a draft checkpoint converted or quantized with a tool that omitted embedding weights; mixing draft/target checkpoints of different revisions; custom Nemotron draft model implementations that never set the embedding module.","solutions":["Verify the draft checkpoint actually contains the embedding weights (inspect the safetensors index for model.embed_tokens.weight) and that files aren't truncated","Check the draft model's get_input_embeddings() implementation returns the real nn.Embedding module (not None due to tying/config flags)","Re-download or re-convert the draft checkpoint with the embedding included","If embeddings are intentionally tied, fix the model class so get_input_embeddings resolves the tied weight"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"draft = worker.draft_worker.model\nif getattr(draft, \"is_nemotron_35_draft\", False):\n    assert draft.get_input_embeddings() is not None, (\n        \"Nemotron 3.5 draft checkpoint is missing its embedding weights\"\n    )","typeGuard":null,"tryCatchPattern":"try:\n    worker = DFlashWorker(...)\nexcept RuntimeError as e:\n    if \"requires its checkpoint embedding\" in str(e):\n        log.error(\"Draft checkpoint incomplete; re-download the draft weights\")\n        raise","preventionTips":["Validate draft checkpoint safetensors contain embedding weights before launch","Load target and draft from the same official release bundle","Smoke-test model loading in CI before long serving runs"],"tags":["sglang","speculative-decoding","dflash","nemotron","checkpoint","embedding"],"backgroundTag":"missing-checkpoint-weights","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}