sgl-project/sglang · error · ValueError

kv-canary: VerifyPlan verify_capacity must be positive, got

Error message

kv-canary: VerifyPlan verify_capacity must be positive, got {verify_capacity}

What it means

VerifyPlan.allocate validates that verify_capacity is a positive integer before allocating the verify_slot_indices tensor. A zero or negative capacity is meaningless (there would be no slots to verify) so the factory rejects it immediately. This is a caller-side programming/argument error, not a runtime device failure.

Source

Thrown at python/sglang/kernels/ops/kv_canary/verify.py:203

            "predecessor == previous array entry" assumption.
        verify_num_valid: Active entry count, shape [1], int32. Clamped by the plan kernel to
            min(total_requested, verify_capacity) so the verify kernel grid never reads past the buffer.
        enable: Run-this-step flag, shape [1], int32. 1 = verify kernel runs as usual; 0 = the plan kernel
            detected overflow (requested > verify_capacity) and the entire verify launch is skipped this step.
            Allocated as 1 by default; the plan kernel rewrites it every step.
    """

    verify_slot_indices: torch.Tensor
    verify_expected_tokens: torch.Tensor
    verify_expected_positions: torch.Tensor
    verify_prev_slot_indices: torch.Tensor
    verify_num_valid: torch.Tensor
    enable: torch.Tensor

    @classmethod
    def allocate(cls, *, verify_capacity: int, device: torch.device) -> VerifyPlan:
        if verify_capacity <= 0:
            raise ValueError(
                f"kv-canary: VerifyPlan verify_capacity must be positive, got {verify_capacity}"
            )
        return cls(
            verify_slot_indices=torch.empty(
                verify_capacity, dtype=torch.int64, device=device
            ),
            verify_expected_tokens=torch.empty(
                verify_capacity, dtype=torch.int64, device=device
            ),
            verify_expected_positions=torch.empty(
                verify_capacity, dtype=torch.int64, device=device
            ),
            verify_prev_slot_indices=torch.empty(
                verify_capacity, dtype=torch.int64, device=device
            ),
            verify_num_valid=torch.empty(1, dtype=torch.int32, device=device),
            # enable defaults to 1 ("run verify") so test helpers that build a VerifyPlan
            # directly (no plan kernel) don't have to remember to set it. Plan kernel always

View on GitHub (pinned to 0132848349)

Solutions

1. Check the expression producing verify_capacity and make sure it is >= 1 before calling allocate; skip the verify launch entirely when there is nothing to verify
2. If capacity can legitimately be 0, guard the call site: if verify_capacity > 0: plan = VerifyPlan.allocate(...)
3. Add an assert/clamp (e.g. max(verify_capacity, 1)) only if the downstream kernel can tolerate dummy slots — otherwise treat 0 as a no-op

Example fix

// before
plan = VerifyPlan.allocate(verify_capacity=num_verify_tokens, device=dev)
// after
if num_verify_tokens > 0:
    plan = VerifyPlan.allocate(verify_capacity=num_verify_tokens, device=dev)
else:
    plan = None  # nothing to verify

if verify_capacity <= 0:
    raise ValueError(f"verify_capacity must be positive, got {verify_capacity}")
plan = VerifyPlan.allocate(verify_capacity=verify_capacity, device=dev)

def is_valid_verify_capacity(c: int) -> bool:
    return isinstance(c, int) and c > 0

When it happens

Trigger: Calling VerifyPlan.allocate(verify_capacity=0, device=...) or with a negative capacity, e.g. when capacity is derived from a batch size, seq len, or request count that computed to 0 (empty batch, integer underflow, or a len() of an empty list).

Common situations: Passing num_verify_tokens computed as max(0, x - y) that clipped to 0 on an empty batch; unit tests constructing plans with degenerate capacities; capacity read from config defaulting to 0 before being set.

Related errors

• kv-canary: write_req_capacity must be non-negative, got {wri
• kv-canary: verify_capacity must be non-negative, got {verify
• kv-canary: WritePlan write_req_capacity must be positive, go
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i
• kv-canary: launch_canary_plan_kernels verify_capacity={verif

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/42f2cce0ee87872a. Report an issue: GitHub.