sgl-project/sglang · error · ValueError

kv-canary: launch_canary_plan_kernels verify_capacity={verif

Error message

kv-canary: launch_canary_plan_kernels verify_capacity={verify_capacity} does not match verify_plan_out.verify_slot_indices.shape[0]={plan_verify_capacity}

What it means

The verify capacity scalar passed by the caller must exactly equal the first dimension of verify_plan_out.verify_slot_indices. The plan kernels write into preallocated output buffers whose sizes were derived from that tensor, so a mismatch would cause out-of-bounds or silently wrong plans.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/api.py:127

    bs = int(req_pool_indices.shape[0])
    if bs > _PLAN_BS_BLOCK_SIZE:
        raise ValueError(
            f"kv-canary: launch_canary_plan_kernels supports at most bs={_PLAN_BS_BLOCK_SIZE} reqs per launch, "
            f"got bs={bs}. Bump _PLAN_BS_BLOCK_SIZE if real workloads need this."
        )
    if swa_window_size > 0 and full_to_swa_index_mapping is None:
        raise ValueError(
            "kv-canary: launch_canary_plan_kernels requires full_to_swa_index_mapping when swa_window_size > 0"
        )

    device = verify_plan_out.verify_slot_indices.device
    verify_offsets_scratch = torch.empty(
        _PLAN_BS_BLOCK_SIZE + 1, dtype=torch.int64, device=device
    )

    plan_verify_capacity = int(verify_plan_out.verify_slot_indices.shape[0])
    if verify_capacity != plan_verify_capacity:
        raise ValueError(
            f"kv-canary: launch_canary_plan_kernels verify_capacity={verify_capacity} does not match "
            f"verify_plan_out.verify_slot_indices.shape[0]={plan_verify_capacity}"
        )

    write_plan_out.write_offsets.zero_()

    launch_plan_offsets_kernel(
        req_pool_indices=req_pool_indices,
        prefix_lens=prefix_lens,
        extend_seq_lens=extend_seq_lens,
        req_to_token=req_to_token,
        full_to_swa_index_mapping=full_to_swa_index_mapping,
        out_verify_offsets_scratch=verify_offsets_scratch,
        out_write_offsets=write_plan_out.write_offsets,
        out_write_seed_slot_indices=write_plan_out.write_seed_slot_indices,
        out_verify_num_valid=verify_plan_out.verify_num_valid,
        out_verify_enable=verify_plan_out.enable,
        out_write_num_valid_reqs=write_plan_out.write_num_valid_reqs,

View on GitHub (pinned to 0132848349)

Solutions

1. Set verify_capacity = int(verify_plan_out.verify_slot_indices.shape[0]) at the call site
2. Audit the buffer allocation code to ensure verify_slot_indices is sized by the same pool capacity used for verify_capacity

Example fix

// before
verify_capacity = write_pool_capacity
launch_canary_plan_kernels(..., verify_capacity=verify_capacity, ...)
// after
verify_capacity = int(verify_plan_out.verify_slot_indices.shape[0])
launch_canary_plan_kernels(..., verify_capacity=verify_capacity, ...)

verify_capacity = int(verify_plan_out.verify_slot_indices.shape[0])
assert verify_capacity >= 0

When it happens

Trigger: Calling launch_canary_plan_kernels where verify_capacity != verify_plan_out.verify_slot_indices.shape[0], e.g. capacity computed from a different pool than the one backing verify_slot_indices.

Common situations: Mixing capacities from the write pool and verify pool, or resizing the verify slot buffer after computing verify_capacity; version changes that changed which pool the verify buffer comes from.

Related errors

• kv-canary: launch_canary_plan_kernels requires full_to_swa_i
• kv-canary: launch_plan_entries_kernel requires req_to_verify
• kv-canary: write_offsets_len must be positive, got {write_of
• kv-canary: write_req_capacity must be non-negative, got {wri
• kv-canary: verify_capacity must be non-negative, got {verify

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0189c318e7e52c1b. Report an issue: GitHub.