sgl-project/sglang · error · ValueError

kv-canary: speculative_num_draft_tokens must be non-negative

Error message

kv-canary: speculative_num_draft_tokens must be non-negative, got {spec_num_draft_tokens}

What it means

from_args normalizes speculative_num_draft_tokens (None -> 0) from the speculative config and requires it to be non-negative, since draft-token count feeds per-forward canary capacities. A negative draft-token count is nonsensical and raises ValueError.

Source

Thrown at python/sglang/srt/kv_canary/capacities.py:80

        if pool_slot_count <= 0:
            raise ValueError(
                f"kv-canary: pool_slot_count must be positive, got {pool_slot_count}"
            )

        cuda_graph_config = get_exec().graph.cuda_graph_config
        cuda_graph_max_bs = (
            cuda_graph_config.decode.max_bs if cuda_graph_config is not None else 0
        ) or 0
        if cuda_graph_max_bs < 0:
            raise ValueError(
                f"kv-canary: cuda_graph_max_bs must be non-negative, got {cuda_graph_max_bs}"
            )

        spec_num_draft_tokens = get_spec().speculative_num_draft_tokens
        if spec_num_draft_tokens is None:
            spec_num_draft_tokens = 0
        if spec_num_draft_tokens < 0:
            raise ValueError(
                "kv-canary: speculative_num_draft_tokens must be non-negative, "
                f"got {spec_num_draft_tokens}"
            )

        max_prefill_tokens = get_schedule().max_prefill_tokens
        if max_prefill_tokens <= 0:
            raise ValueError(
                f"kv-canary: max_prefill_tokens must be positive, got {max_prefill_tokens}"
            )

        num_tokens_per_req = 1
        if spec_num_draft_tokens:
            num_tokens_per_req = max(num_tokens_per_req, spec_num_draft_tokens)

        max_bs = max(cuda_graph_max_bs, req_to_token_pool_size)

        chunked_prefill_size = get_schedule().chunked_prefill_size
        chunked_limit = (

View on GitHub (pinned to 0132848349)

Solutions

1. Set speculative_num_draft_tokens to None or 0 when speculative decoding is unused
2. Validate spec config values are non-negative before launching with --kv-canary enabled
3. Regenerate the server args / spec config with the current sglang version instead of reusing stale files

Example fix

# before
ServerArgs(..., speculative_num_draft_tokens=-1, kv_canary='raise')

# after
ServerArgs(..., speculative_num_draft_tokens=None, kv_canary='raise')

n = server_args.speculative_num_draft_tokens
if n is not None and n < 0:
    raise SystemExit("speculative_num_draft_tokens must be None/0 or positive")
# then proceed with install_canary

def valid_draft_tokens(n) -> bool:
    return n is None or (isinstance(n, int) and n >= 0)

When it happens

Trigger: Calling install_canary/from_args when the speculative decoding config carries a negative speculative_num_draft_tokens, e.g. a custom spec algorithm config or server-args override with a negative sentinel value.

Common situations: Scripts building speculative configs programmatically that use -1 as 'unset' instead of None/0; version changes where the draft-token field moved and stale configs supply invalid values.

Understand the failure class

Background: Config validation failed: what "invalid value for {key}" and settings-rejection errors mean across 19 open-source libraries — this error's family across 19 libraries.

Related errors

• kv-canary: {name} must be positive, got {value}
• kv-canary: req_to_token_pool_size must be positive, got {req
• kv-canary: max_seq_len_per_req must be positive, got {max_se
• kv-canary: pool_slot_count must be positive, got {pool_slot_
• kv-canary: cuda_graph_max_bs must be non-negative, got {cuda

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2e054181a5c2928f. Report an issue: GitHub.