sgl-project/sglang · error · ValueError

kv-canary: req_to_token_pool_size must be positive, got {req

Error message

kv-canary: req_to_token_pool_size must be positive, got {req_to_token_pool_size}

What it means

CanaryLaunchCapacities.from_args validates its inputs and requires req_to_token_pool_size to be strictly positive because the canary installs itself into the req-to-token pool. A non-positive pool size means there is no request pool to guard, so the constructor raises ValueError immediately.

Source

Thrown at python/sglang/srt/kv_canary/capacities.py:53

    def __post_init__(self) -> None:
        for name, value in (
            ("per_forward_verify_capacity", self.per_forward_verify_capacity),
            ("per_forward_write_req_capacity", self.per_forward_write_req_capacity),
            ("per_forward_write_entry_capacity", self.per_forward_write_entry_capacity),
        ):
            if value <= 0:
                raise ValueError(f"kv-canary: {name} must be positive, got {value}")

    @classmethod
    def from_args(
        cls,
        *,
        req_to_token_pool_size: int,
        max_seq_len_per_req: int,
        pool_slot_count: int,
    ) -> CanaryLaunchCapacities:
        if req_to_token_pool_size <= 0:
            raise ValueError(
                "kv-canary: req_to_token_pool_size must be positive, "
                f"got {req_to_token_pool_size}"
            )
        if max_seq_len_per_req <= 0:
            raise ValueError(
                "kv-canary: max_seq_len_per_req must be positive, "
                f"got {max_seq_len_per_req}"
            )
        if pool_slot_count <= 0:
            raise ValueError(
                f"kv-canary: pool_slot_count must be positive, got {pool_slot_count}"
            )

        cuda_graph_config = get_exec().graph.cuda_graph_config
        cuda_graph_max_bs = (
            cuda_graph_config.decode.max_bs if cuda_graph_config is not None else 0
        ) or 0
        if cuda_graph_max_bs < 0:

View on GitHub (pinned to 0132848349)

Solutions

1. Pass a positive req_to_token_pool_size (e.g. the server's actual req-to-token pool size)
2. If canary is not wanted, run with --kv-canary none instead of zeroing pool sizes
3. Audit scripts that compute pool sizes to guarantee a minimum of 1

Example fix

# before
--max-running-requests 0 --kv-canary raise

# after
--max-running-requests 256 --kv-canary raise

if req_to_token_pool_size <= 0:
    raise SystemExit("req_to_token_pool_size must be > 0; check --max-running-requests")
caps = CanaryLaunchCapacities.from_args(
    req_to_token_pool_size=req_to_token_pool_size, ...
)

def is_valid_pool_size(n: int) -> bool:
    return isinstance(n, int) and n > 0

When it happens

Trigger: Calling from_args (via install_canary) with req_to_token_pool_size <= 0, typically because the server was started with --max-running-requests 0 or a req-to-token pool sized to zero by another flag.

Common situations: Setting --max-running-requests 0 or a pool-override flag to 0 while enabling --kv-canary log/raise; automation scripts that derive pool size arithmetically and can produce 0.

Understand the failure class

Background: Config validation failed: what "invalid value for {key}" and settings-rejection errors mean across 19 open-source libraries — this error's family across 19 libraries.

Related errors

• kv-canary: {name} must be positive, got {value}
• kv-canary: max_seq_len_per_req must be positive, got {max_se
• kv-canary: pool_slot_count must be positive, got {pool_slot_
• kv-canary: cuda_graph_max_bs must be non-negative, got {cuda
• kv-canary: speculative_num_draft_tokens must be non-negative

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/90039eb3a4590daa. Report an issue: GitHub.