{"record":{"id":"6d46981c2eb2f8c3","repo":"sgl-project/sglang","slug":"unknown-speculative-phase-phase","errorCode":null,"errorMessage":"Unknown speculative phase: {phase}","messagePattern":"Unknown speculative phase: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/spec_utils.py","lineNumber":124,"sourceCode":"    lives on ``SpecInput.num_tokens_per_req``. Draft phases are\n    EAGLE-family-only; \"target_verify\" is algorithm-generic via the hook.\n\n    The widths come from the bags: adaptive spec captures each candidate step\n    config with that config's leaves overridden, so the buffers being sized\n    must follow the override rather than the startup values.\n    \"\"\"\n    spec = get_spec()\n    if phase == \"draft_decode\":\n        return spec.speculative_eagle_topk\n    if phase == \"draft_extend\":\n        return spec.speculative_num_draft_tokens\n    if phase == \"target_verify\":\n        if num_draft_tokens is None:\n            num_draft_tokens = spec.speculative_num_draft_tokens\n        return spec_algorithm.get_num_tokens_per_req_for_target_verify(\n            num_draft_tokens, is_draft_worker\n        )\n    raise ValueError(f\"Unknown speculative phase: {phase}\")\n\n\ndef fast_sample(probs: torch.Tensor, num_samples: int = 1):\n    \"\"\"Draw from `probs` via the Gumbel-max trick: argmax(probs / Exp(1)).\n\n    Distributionally equivalent to torch.multinomial, but avoids multinomial's\n    device-side distribution-validity assert, which the draft CUDA graph would\n    otherwise capture and replay every step. q is clamped off zero so a zero\n    draw can't yield inf/NaN scores that argmax would wrongly select; fp32\n    avoids bf16 argmax ties biasing the draw. Set SGLANG_OPT_USE_GUMBEL_SAMPLE=0\n    to fall back to torch.multinomial.\n    \"\"\"\n    if not envs.SGLANG_OPT_USE_GUMBEL_SAMPLE.get():\n        sample_index = torch.multinomial(probs, num_samples=num_samples)\n        return probs.gather(1, sample_index), sample_index\n    q = torch.empty_like(probs, dtype=torch.float32).exponential_(1.0)\n    q.clamp_min_(torch.finfo(torch.float32).tiny)\n    scores = probs.float() / q","sourceCodeStart":106,"sourceCodeEnd":142,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/spec_utils.py#L106-L142","documentation":"resolve_num_tokens_per_req dispatches per speculative phase; only 'decode' and 'target_verify' are recognized. Any other phase string falls through to ValueError('Unknown speculative phase').","triggerScenarios":"Calling resolve_num_tokens_per_req(phase='draft') or a typo like 'target-verify'; the function is used both in worker __init__ and decode_num_tokens_per_req.","commonSituations":"New code paths (e.g. draft-worker extend phase) passing an unhandled phase; typos or refactor renaming phases without updating call sites.","solutions":["Use only 'decode' or 'target_verify'","If you need a new phase, extend the if/elif chain in spec_utils.py","Check the call site spelling and constants used for phase"],"exampleFix":"# before\nn = resolve_num_tokens_per_req(spec, phase='target-verify')\n# after\nn = resolve_num_tokens_per_req(spec, phase='target_verify')","handlingStrategy":"validation","validationCode":"assert phase in ('decode', 'target_verify'), phase\nn = resolve_num_tokens_per_req(spec, phase=phase, ...)","typeGuard":"def is_valid_phase(phase: str) -> bool:\n    return phase in ('decode', 'target_verify')","tryCatchPattern":null,"preventionTips":["Centralize phase strings as module constants instead of inline literals"],"tags":["speculative-decoding","validation","enum-lookup"],"backgroundTag":"invalid-enum-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}