sgl-project/sglang · error · ValueError

kv-canary: cuda_graph_max_bs must be non-negative, got {cuda

Error message

kv-canary: cuda_graph_max_bs must be non-negative, got {cuda_graph_max_bs}

What it means

from_args reads the CUDA graph decode max batch size from the runtime config (defaulting to 0 when graphs are disabled) and requires it to be non-negative. A negative value would corrupt the canary's batch-size capacity math, so it raises ValueError with the offending number.

Source

Thrown at python/sglang/srt/kv_canary/capacities.py:72

                "kv-canary: req_to_token_pool_size must be positive, "
                f"got {req_to_token_pool_size}"
            )
        if max_seq_len_per_req <= 0:
            raise ValueError(
                "kv-canary: max_seq_len_per_req must be positive, "
                f"got {max_seq_len_per_req}"
            )
        if pool_slot_count <= 0:
            raise ValueError(
                f"kv-canary: pool_slot_count must be positive, got {pool_slot_count}"
            )

        cuda_graph_config = get_exec().graph.cuda_graph_config
        cuda_graph_max_bs = (
            cuda_graph_config.decode.max_bs if cuda_graph_config is not None else 0
        ) or 0
        if cuda_graph_max_bs < 0:
            raise ValueError(
                f"kv-canary: cuda_graph_max_bs must be non-negative, got {cuda_graph_max_bs}"
            )

        spec_num_draft_tokens = get_spec().speculative_num_draft_tokens
        if spec_num_draft_tokens is None:
            spec_num_draft_tokens = 0
        if spec_num_draft_tokens < 0:
            raise ValueError(
                "kv-canary: speculative_num_draft_tokens must be non-negative, "
                f"got {spec_num_draft_tokens}"
            )

        max_prefill_tokens = get_schedule().max_prefill_tokens
        if max_prefill_tokens <= 0:
            raise ValueError(
                f"kv-canary: max_prefill_tokens must be positive, got {max_prefill_tokens}"
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Use 0 (or omit the flag) to disable CUDA graphs instead of a negative value
2. Validate/normalize cuda_graph_max_bs >= 0 before building configs in custom launch code
3. Check the constructed cuda_graph_config.decode.max_bs in a debugger/repro script

Example fix

# before
--cuda-graph-max-bs -1

# after
--cuda-graph-max-bs 0  # or omit to use defaults

cfg = get_exec().graph.cuda_graph_config
max_bs = (cfg.decode.max_bs if cfg is not None else 0) or 0
if max_bs < 0:
    raise SystemExit("cuda_graph_max_bs must be >= 0; use 0 to disable graphs")
caps = CanaryLaunchCapacities.from_args(...)

def is_non_negative(n) -> bool:
    return isinstance(n, int) and n >= 0

When it happens

Trigger: Calling install_canary/from_args when cuda_graph_config.decode.max_bs is negative — only possible via a programmatically built CUDA graph config or a server-args override that sets a negative max batch size (e.g. --cuda-graph-max-bs -1 reaching the config unvalidated).

Common situations: Test harnesses or scripts constructing graph configs manually with invalid negative max_bs; flags like --cuda-graph-max-bs set to negative sentinel values meaning 'disabled' where 0 is expected.

Understand the failure class

Background: Config validation failed: what "invalid value for {key}" and settings-rejection errors mean across 19 open-source libraries — this error's family across 19 libraries.

Related errors

• kv-canary: {name} must be positive, got {value}
• kv-canary: req_to_token_pool_size must be positive, got {req
• kv-canary: max_seq_len_per_req must be positive, got {max_se
• kv-canary: pool_slot_count must be positive, got {pool_slot_
• kv-canary: speculative_num_draft_tokens must be non-negative

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a3f43f4285d19ad5. Report an issue: GitHub.