sgl-project/sglang · error · ValueError

kv-canary: write_offsets_len must equal write_req_capacity +

Error message

kv-canary: write_offsets_len must equal write_req_capacity + 1, got {write_offsets_len} and {write_req_capacity}

What it means

The write-offsets array is a prefix-sum layout: it must hold exactly write_req_capacity + 1 int64 entries (one leading zero plus one offset per possible request). Any other length breaks the kernel's scatter/gather indexing.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/offsets_kernel.py:205

    _require_min_len(lut_tensor, "lut_tensor", max(lut_len, 1))
    _require_min_len(
        out_verify_offsets_scratch,
        "out_verify_offsets_scratch",
        _PLAN_BS_BLOCK_SIZE + 1,
    )
    _require_len(out_write_offsets, "out_write_offsets", write_offsets_len)
    _require_len(
        out_write_seed_slot_indices,
        "out_write_seed_slot_indices",
        write_req_capacity,
    )
    _require_len(out_verify_num_valid, "out_verify_num_valid", 1)
    _require_len(out_verify_enable, "out_verify_enable", 1)
    _require_len(out_write_num_valid_reqs, "out_write_num_valid_reqs", 1)
    _require_1d(lut_tensor, "lut_tensor")

    if write_offsets_len != write_req_capacity + 1:
        raise ValueError(
            f"kv-canary: write_offsets_len must equal write_req_capacity + 1, got "
            f"{write_offsets_len} and {write_req_capacity}"
        )
    if bs > write_req_capacity:
        raise ValueError(
            f"kv-canary: bs={bs} exceeds write_req_capacity={write_req_capacity}"
        )
    if req_to_token_stride0 != int(req_to_token.stride(0)):
        raise ValueError(
            f"kv-canary: req_to_token_stride0={req_to_token_stride0} does not match "
            f"req_to_token.stride(0)={int(req_to_token.stride(0))}"
        )

    _require_same_device(
        out_verify_offsets_scratch,
        "out_verify_offsets_scratch",
        (
            (req_pool_indices, "req_pool_indices"),

View on GitHub (pinned to 0132848349)

Solutions

1. Allocate write_offsets = torch.empty(write_req_capacity + 1, dtype=torch.int64, device=dev) and pass write_offsets_len = write_req_capacity + 1
2. Add an assertion at allocation time to keep the invariant explicit

Example fix

// before
write_offsets = torch.zeros(write_req_capacity, dtype=torch.int64, device=dev)
launch_plan_offsets_kernel(..., write_offsets_len=write_offsets.shape[0], ...)
// after
write_offsets = torch.zeros(write_req_capacity + 1, dtype=torch.int64, device=dev)
launch_plan_offsets_kernel(..., write_offsets_len=write_offsets.shape[0], ...)

assert write_offsets_len == write_req_capacity + 1

When it happens

Trigger: Calling launch_plan_offsets_kernel where write_offsets_len != write_req_capacity + 1 — classically allocating write_offsets with write_req_capacity elements.

Common situations: Off-by-one when allocating the offsets buffer; reusing a buffer sized for a different pool capacity after resizing.

Related errors

• kv-canary: write_offsets_len must be positive, got {write_of
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i
• kv-canary: launch_canary_plan_kernels verify_capacity={verif
• kv-canary: launch_plan_entries_kernel requires req_to_verify
• kv-canary: offsets kernel bs must be in [0, {_PLAN_BS_BLOCK_

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/62ed8a67086dfca2. Report an issue: GitHub.