sgl-project/sglang · error · ValueError

kv-canary: expected input tensors must be None when enable_w

Error message

kv-canary: expected input tensors must be None when enable_write_input_assert=False

What it means

With enable_write_input_assert=False the reference write kernel has no consumer for expected tensors, so supplying expected_input_tokens or expected_input_positions is treated as a configuration inconsistency and rejected, mirroring the CUDA launcher's check exactly.

Source

Thrown at python/sglang/kernels/ops/kv_canary/write_ref.py:93

    if slot_stride_i64 < 4:
        raise ValueError(
            f"kv-canary: canary_buf slot stride must hold at least 4 int64 fields, got {slot_stride_i64}"
        )

    if enable_write_input_assert:
        if expected_input_tokens is None or expected_input_positions is None:
            raise ValueError(
                "kv-canary: expected input tensors are required when enable_write_input_assert=True"
            )
        expected_input_tokens_host = expected_input_tokens.detach().to(
            device=work_device, dtype=torch.int64
        )
        expected_input_positions_host = expected_input_positions.detach().to(
            device=work_device, dtype=torch.int64
        )
    else:
        if expected_input_tokens is not None or expected_input_positions is not None:
            raise ValueError(
                "kv-canary: expected input tensors must be None when enable_write_input_assert=False"
            )
        expected_input_tokens_host = None
        expected_input_positions_host = None

    violation_rows: list[list[int]] = []
    total_slots_written = 0

    for r in range(active_reqs):
        entry_start = int(write_offsets_host[r].item())
        entry_end = int(write_offsets_host[r + 1].item())
        entry_count = entry_end - entry_start
        if entry_count <= 0:
            continue

        seed_slot = int(seed_slot_indices_host[r].item())
        running_prev_hash = compute_slot_hash(buf_i64, seed_slot)

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the expected_* arguments when the flag is False
2. Or flip enable_write_input_assert to True to actually use them
3. Gate forwarding of expected tensors on the flag in wrappers

Example fix

# before
launch_canary_write_kernel_torch_reference(ctx, plan, ..., enable_write_input_assert=False,
    expected_input_tokens=t)
# after
launch_canary_write_kernel_torch_reference(ctx, plan, ..., enable_write_input_assert=False)

if not enable_write_input_assert:
    expected_input_tokens = expected_input_positions = None

def ref_assert_args_ok(flag: bool, t, p) -> bool:
    return (flag and t is not None and p is not None) or (not flag and t is None and p is None)

When it happens

Trigger: Calling launch_canary_write_kernel_torch_reference(..., enable_write_input_assert=False) while passing either expected tensor.

Common situations: Reusing a debug call site with tensors attached after flipping the flag off; generic wrappers that always forward expected tensors regardless of the flag.

Related errors

• kv-canary: canary_buf slot stride must hold at least 4 int64
• kv-canary: expected input tensors must be None when enable_w
• kv-canary: canary_buf slot stride must hold at least 4 int64
• kv-canary: expected input tensors are required when enable_w
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2639f00eff7694fb. Report an issue: GitHub.