sgl-project/sglang · error · ValueError

kv-canary: max_prefill_tokens must be positive, got {max_pre

Error message

kv-canary: max_prefill_tokens must be positive, got {max_prefill_tokens}

What it means

from_args requires the scheduler's max_prefill_tokens to be strictly positive because the canary sizes its per-forward buffers from the prefill token budget. A zero or negative prefill budget means no prefill batch can ever run, so the canary refuses to install.

Source

Thrown at python/sglang/srt/kv_canary/capacities.py:87

            cuda_graph_config.decode.max_bs if cuda_graph_config is not None else 0
        ) or 0
        if cuda_graph_max_bs < 0:
            raise ValueError(
                f"kv-canary: cuda_graph_max_bs must be non-negative, got {cuda_graph_max_bs}"
            )

        spec_num_draft_tokens = get_spec().speculative_num_draft_tokens
        if spec_num_draft_tokens is None:
            spec_num_draft_tokens = 0
        if spec_num_draft_tokens < 0:
            raise ValueError(
                "kv-canary: speculative_num_draft_tokens must be non-negative, "
                f"got {spec_num_draft_tokens}"
            )

        max_prefill_tokens = get_schedule().max_prefill_tokens
        if max_prefill_tokens <= 0:
            raise ValueError(
                f"kv-canary: max_prefill_tokens must be positive, got {max_prefill_tokens}"
            )

        num_tokens_per_req = 1
        if spec_num_draft_tokens:
            num_tokens_per_req = max(num_tokens_per_req, spec_num_draft_tokens)

        max_bs = max(cuda_graph_max_bs, req_to_token_pool_size)

        chunked_prefill_size = get_schedule().chunked_prefill_size
        chunked_limit = (
            chunked_prefill_size
            if chunked_prefill_size is not None and chunked_prefill_size >= 0
            else math.inf
        )
        max_extend_tokens_per_forward = min(max_prefill_tokens, chunked_limit)

        write_entry_capacity = max(

View on GitHub (pinned to 0132848349)

Solutions

1. Set --max-prefill-tokens to a positive value (e.g. 16384)
2. Remove the flag to use the default scheduler budget
3. Run with --kv-canary none if you intentionally want zero prefill budget

Example fix

# before
--max-prefill-tokens 0 --kv-canary log

# after
--max-prefill-tokens 16384 --kv-canary log

if get_schedule().max_prefill_tokens <= 0:
    raise SystemExit("max_prefill_tokens must be > 0; set --max-prefill-tokens 16384 or remove the flag")
caps = CanaryLaunchCapacities.from_args(...)

def valid_prefill_budget(n: int) -> bool:
    return isinstance(n, int) and n > 0

When it happens

Trigger: Calling install_canary/from_args with --max-prefill-tokens 0 or negative, or a scheduler config where max_prefill_tokens was overridden/computed to <= 0.

Common situations: Explicitly setting --max-prefill-tokens 0 in benchmarking or memory-constrained setups and forgetting to disable --kv-canary; derived configs in test harnesses that zero out scheduler budgets.

Understand the failure class

Background: Config validation failed: what "invalid value for {key}" and settings-rejection errors mean across 19 open-source libraries — this error's family across 19 libraries.

Related errors

• kv-canary: {name} must be positive, got {value}
• kv-canary: req_to_token_pool_size must be positive, got {req
• kv-canary: max_seq_len_per_req must be positive, got {max_se
• kv-canary: pool_slot_count must be positive, got {pool_slot_
• kv-canary: cuda_graph_max_bs must be non-negative, got {cuda

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/822c4b6e65ea3139. Report an issue: GitHub.