sgl-project/sglang · error · ValueError

kv-canary: offsets kernel bs must be in [0, {_PLAN_BS_BLOCK_

Error message

kv-canary: offsets kernel bs must be in [0, {_PLAN_BS_BLOCK_SIZE}], got {bs}

What it means

The offsets kernel is compiled for a fixed batch-size block, so the runtime batch size must be within [0, _PLAN_BS_BLOCK_SIZE]. Values outside that range would break kernel indexing assumptions.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/offsets_kernel.py:153

    _require_dtype(req_pool_indices, "req_pool_indices", torch.int64)
    _require_dtype(prefix_lens, "prefix_lens", torch.int64)
    _require_dtype(extend_seq_lens, "extend_seq_lens", torch.int64)
    _require_dtype(req_to_token, "req_to_token", torch.int32)
    _require_dtype(lut_tensor, "lut_tensor", torch.int64)
    _require_dtype(
        out_verify_offsets_scratch, "out_verify_offsets_scratch", torch.int64
    )
    _require_dtype(out_write_offsets, "out_write_offsets", torch.int64)
    _require_dtype(
        out_write_seed_slot_indices, "out_write_seed_slot_indices", torch.int64
    )
    _require_dtype(out_verify_num_valid, "out_verify_num_valid", torch.int32)
    _require_dtype(out_verify_enable, "out_verify_enable", torch.int32)
    _require_dtype(out_write_num_valid_reqs, "out_write_num_valid_reqs", torch.int32)

    if bs < 0 or bs > _PLAN_BS_BLOCK_SIZE:
        raise ValueError(
            f"kv-canary: offsets kernel bs must be in [0, {_PLAN_BS_BLOCK_SIZE}], got {bs}"
        )
    if write_offsets_len <= 0:
        raise ValueError(
            f"kv-canary: write_offsets_len must be positive, got {write_offsets_len}"
        )
    if write_req_capacity < 0:
        raise ValueError(
            f"kv-canary: write_req_capacity must be non-negative, got {write_req_capacity}"
        )
    if verify_capacity < 0:
        raise ValueError(
            f"kv-canary: verify_capacity must be non-negative, got {verify_capacity}"
        )
    if req_to_token_stride0 <= 0:
        raise ValueError(
            f"kv-canary: req_to_token_stride0 must be positive, got {req_to_token_stride0}"
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Split the launch into chunks of at most _PLAN_BS_BLOCK_SIZE requests
2. Bump _PLAN_BS_BLOCK_SIZE in the kernel module if real workloads need larger blocks (as the sibling error message suggests)
3. Check upstream slicing logic if bs is negative

Example fix

// before
launch_plan_offsets_kernel(bs=len(reqs), ...)
// after
for i in range(0, len(reqs), _PLAN_BS_BLOCK_SIZE):
    launch_plan_offsets_kernel(bs=min(_PLAN_BS_BLOCK_SIZE, len(reqs)-i), ...)

from python.sglang.kernels.ops.kv_canary.plan.offsets_kernel import _PLAN_BS_BLOCK_SIZE
assert 0 <= bs <= _PLAN_BS_BLOCK_SIZE, f"bs={bs} exceeds kernel block {_PLAN_BS_BLOCK_SIZE}"

When it happens

Trigger: Calling launch_plan_offsets_kernel with bs < 0 or bs > _PLAN_BS_BLOCK_SIZE (e.g. a running-batch larger than the compiled block, typically 256).

Common situations: High-concurrency serving where the running batch exceeds the kernel's fixed block size; negative bs from a buggy upstream trim/slice.

Related errors

• kv-canary: bs={bs} exceeds write_req_capacity={write_req_cap
• kv-canary: forward_batch.batch_size={bs} exceeds pre-allocat
• This browser does not support gzip stream decoding
• cos/sin shape does not cover image tokens and head_dim
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/969728e3113b7acd. Report an issue: GitHub.