sgl-project/sglang · error · ValueError

Unknown speculative phase: {phase}

Error message

Unknown speculative phase: {phase}

What it means

resolve_num_tokens_per_req dispatches per speculative phase; only 'decode' and 'target_verify' are recognized. Any other phase string falls through to ValueError('Unknown speculative phase').

Source

Thrown at python/sglang/srt/speculative/spec_utils.py:124

    lives on ``SpecInput.num_tokens_per_req``. Draft phases are
    EAGLE-family-only; "target_verify" is algorithm-generic via the hook.

    The widths come from the bags: adaptive spec captures each candidate step
    config with that config's leaves overridden, so the buffers being sized
    must follow the override rather than the startup values.
    """
    spec = get_spec()
    if phase == "draft_decode":
        return spec.speculative_eagle_topk
    if phase == "draft_extend":
        return spec.speculative_num_draft_tokens
    if phase == "target_verify":
        if num_draft_tokens is None:
            num_draft_tokens = spec.speculative_num_draft_tokens
        return spec_algorithm.get_num_tokens_per_req_for_target_verify(
            num_draft_tokens, is_draft_worker
        )
    raise ValueError(f"Unknown speculative phase: {phase}")


def fast_sample(probs: torch.Tensor, num_samples: int = 1):
    """Draw from `probs` via the Gumbel-max trick: argmax(probs / Exp(1)).

    Distributionally equivalent to torch.multinomial, but avoids multinomial's
    device-side distribution-validity assert, which the draft CUDA graph would
    otherwise capture and replay every step. q is clamped off zero so a zero
    draw can't yield inf/NaN scores that argmax would wrongly select; fp32
    avoids bf16 argmax ties biasing the draw. Set SGLANG_OPT_USE_GUMBEL_SAMPLE=0
    to fall back to torch.multinomial.
    """
    if not envs.SGLANG_OPT_USE_GUMBEL_SAMPLE.get():
        sample_index = torch.multinomial(probs, num_samples=num_samples)
        return probs.gather(1, sample_index), sample_index
    q = torch.empty_like(probs, dtype=torch.float32).exponential_(1.0)
    q.clamp_min_(torch.finfo(torch.float32).tiny)
    scores = probs.float() / q

View on GitHub (pinned to 0132848349)

Solutions

1. Use only 'decode' or 'target_verify'
2. If you need a new phase, extend the if/elif chain in spec_utils.py
3. Check the call site spelling and constants used for phase

Example fix

# before
n = resolve_num_tokens_per_req(spec, phase='target-verify')
# after
n = resolve_num_tokens_per_req(spec, phase='target_verify')

assert phase in ('decode', 'target_verify'), phase
n = resolve_num_tokens_per_req(spec, phase=phase, ...)

def is_valid_phase(phase: str) -> bool:
    return phase in ('decode', 'target_verify')

When it happens

Trigger: Calling resolve_num_tokens_per_req(phase='draft') or a typo like 'target-verify'; the function is used both in worker __init__ and decode_num_tokens_per_req.

Common situations: New code paths (e.g. draft-worker extend phase) passing an unhandled phase; typos or refactor renaming phases without updating call sites.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• kv-canary: launch_canary_plan_kernels_torch_reference verify
• --speculative-ngram-external-sam-budget must be positive whe
• --speculative-ngram-external-corpus-max-tokens must be posit
• speculative_ngram_external_sam_budget must be less than or e
• kv-canary: speculative_num_draft_tokens must be non-negative

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6d46981c2eb2f8c3. Report an issue: GitHub.