sgl-project/sglang · error · ValueError

kv-canary: bs={bs} exceeds write_req_capacity={write_req_cap

Error message

kv-canary: bs={bs} exceeds write_req_capacity={write_req_capacity}

What it means

The offsets kernel writes one offset per request, so the batch size cannot exceed the write-request capacity of the output buffers. bs > write_req_capacity would overflow the write_offsets/plan buffers.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/offsets_kernel.py:210

    )
    _require_len(out_write_offsets, "out_write_offsets", write_offsets_len)
    _require_len(
        out_write_seed_slot_indices,
        "out_write_seed_slot_indices",
        write_req_capacity,
    )
    _require_len(out_verify_num_valid, "out_verify_num_valid", 1)
    _require_len(out_verify_enable, "out_verify_enable", 1)
    _require_len(out_write_num_valid_reqs, "out_write_num_valid_reqs", 1)
    _require_1d(lut_tensor, "lut_tensor")

    if write_offsets_len != write_req_capacity + 1:
        raise ValueError(
            f"kv-canary: write_offsets_len must equal write_req_capacity + 1, got "
            f"{write_offsets_len} and {write_req_capacity}"
        )
    if bs > write_req_capacity:
        raise ValueError(
            f"kv-canary: bs={bs} exceeds write_req_capacity={write_req_capacity}"
        )
    if req_to_token_stride0 != int(req_to_token.stride(0)):
        raise ValueError(
            f"kv-canary: req_to_token_stride0={req_to_token_stride0} does not match "
            f"req_to_token.stride(0)={int(req_to_token.stride(0))}"
        )

    _require_same_device(
        out_verify_offsets_scratch,
        "out_verify_offsets_scratch",
        (
            (req_pool_indices, "req_pool_indices"),
            (prefix_lens, "prefix_lens"),
            (extend_seq_lens, "extend_seq_lens"),
            (req_to_token, "req_to_token"),
            (lut_tensor, "lut_tensor"),
            (out_write_offsets, "out_write_offsets"),

View on GitHub (pinned to 0132848349)

Solutions

1. Clamp/chunk the launch so bs <= write_req_capacity, splitting into multiple launches
2. Grow write_req_capacity (and the buffers sized capacity+1) to cover the max running batch

Example fix

// before
launch_plan_offsets_kernel(bs=len(reqs), write_req_capacity=cap, ...)
// after
assert len(reqs) <= cap
launch_plan_offsets_kernel(bs=len(reqs), write_req_capacity=cap, ...)

assert bs <= write_req_capacity, f"bs={bs} > write_req_capacity={write_req_capacity}"

When it happens

Trigger: Calling launch_plan_offsets_kernel with a bs larger than write_req_capacity — e.g. a scheduler batch that grew past the plan buffer's capacity.

Common situations: Running batch grows after the plan buffers were allocated for a smaller max; chunked-prefill misconfiguration inflating batch size; capacity reduced by other pools.

Related errors

• kv-canary: offsets kernel bs must be in [0, {_PLAN_BS_BLOCK_
• kv-canary: write_req_capacity must be non-negative, got {wri
• kv-canary: verify_capacity must be non-negative, got {verify
• kv-canary: VerifyPlan verify_capacity must be positive, got
• kv-canary: WritePlan write_req_capacity must be positive, go

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/548b0edd023ac9fd. Report an issue: GitHub.