sgl-project/sglang · error · ValueError

kv-canary: launch_canary_plan_kernels requires full_to_swa_i

Error message

kv-canary: launch_canary_plan_kernels requires full_to_swa_index_mapping when swa_window_size > 0

What it means

The kv-canary plan launcher requires a full-to-SWA index mapping (LUT) whenever a positive sliding-window size is given, because the SWA path needs the mapping to translate full KV indices into SWA-pool indices. Omitting it would make the kernel unable to compute SWA offsets safely, so it fails fast with a ValueError.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/api.py:116

    Calling contract:
        - Pure side-effect; no host work, no D2H.
        - Safe in cuda-graph capture; caller refills all input tensors in-place before replay.
        - The wrapper launches the plan sub-kernels needed to fill both plans end-to-end.
        - Padding rows contribute zero entries.

    Pinned by Python reference
    :func:`sglang.kernels.ops.kv_canary.plan_ref.launch_canary_plan_kernels_torch_reference`; both the Triton
    offsets kernel and the CUDA JIT entries kernel must match byte-for-byte.
    """
    bs = int(req_pool_indices.shape[0])
    if bs > _PLAN_BS_BLOCK_SIZE:
        raise ValueError(
            f"kv-canary: launch_canary_plan_kernels supports at most bs={_PLAN_BS_BLOCK_SIZE} reqs per launch, "
            f"got bs={bs}. Bump _PLAN_BS_BLOCK_SIZE if real workloads need this."
        )
    if swa_window_size > 0 and full_to_swa_index_mapping is None:
        raise ValueError(
            "kv-canary: launch_canary_plan_kernels requires full_to_swa_index_mapping when swa_window_size > 0"
        )

    device = verify_plan_out.verify_slot_indices.device
    verify_offsets_scratch = torch.empty(
        _PLAN_BS_BLOCK_SIZE + 1, dtype=torch.int64, device=device
    )

    plan_verify_capacity = int(verify_plan_out.verify_slot_indices.shape[0])
    if verify_capacity != plan_verify_capacity:
        raise ValueError(
            f"kv-canary: launch_canary_plan_kernels verify_capacity={verify_capacity} does not match "
            f"verify_plan_out.verify_slot_indices.shape[0]={plan_verify_capacity}"
        )

    write_plan_out.write_offsets.zero_()

    launch_plan_offsets_kernel(

View on GitHub (pinned to 0132848349)

Solutions

1. Pass the full_to_swa_index_mapping tensor produced by your SWA/hybrid KV allocator when swa_window_size > 0
2. If your model has no SWA layers, pass swa_window_size=0 so no mapping is required
3. Check the caller (invoke_plan/_run_both_plan) to confirm the mapping is forwarded from the memory pool config

Example fix

// before
launch_canary_plan_kernels(..., swa_window_size=window, full_to_swa_index_mapping=None)
// after
launch_canary_plan_kernels(..., swa_window_size=window, full_to_swa_index_mapping=full_to_swa_lut)

if swa_window_size > 0:
    assert full_to_swa_index_mapping is not None, "SWA window requires full_to_swa_index_mapping"

def has_swa_mapping(swa_window_size: int, lut: torch.Tensor | None) -> bool:
    return swa_window_size <= 0 or lut is not None

When it happens

Trigger: Calling launch_canary_plan_kernels (directly or via invoke_plan/_run_both_plan) with swa_window_size > 0 but full_to_swa_index_mapping=None.

Common situations: Running a hybrid SWA model (e.g. Gemma-2/3-style sliding window) where the caller wired swa_window_size from the attention backend but forgot to pass the mapping tensor built by the memory pool/token allocator.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• kv-canary: launch_canary_plan_kernels verify_capacity={verif
• kv-canary: launch_plan_entries_kernel requires req_to_verify
• kv-canary: write_offsets_len must be positive, got {write_of
• kv-canary: write_req_capacity must be non-negative, got {wri
• kv-canary: verify_capacity must be non-negative, got {verify

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/00da68969d408fef. Report an issue: GitHub.