sgl-project/sglang · error · ValueError

kv-canary: expected input tensors are required when enable_w

Error message

kv-canary: expected input tensors are required when enable_write_input_assert=True

What it means

The torch reference write path validates the optional input-assert mode: with enable_write_input_assert=True both expected_input_tokens and expected_input_positions must be supplied so the reference can emulate the kernel's cross-check. Missing tensors make the requested mode impossible.

Source

Thrown at python/sglang/kernels/ops/kv_canary/write_ref.py:82

    if total_entries <= 0:
        return

    buf_i64 = (
        canary_buf.detach()
        .to(device=work_device)
        .contiguous()
        .view(torch.int64)
        .clone()
    )
    slot_stride_i64 = int(buf_i64.shape[1])
    if slot_stride_i64 < 4:
        raise ValueError(
            f"kv-canary: canary_buf slot stride must hold at least 4 int64 fields, got {slot_stride_i64}"
        )

    if enable_write_input_assert:
        if expected_input_tokens is None or expected_input_positions is None:
            raise ValueError(
                "kv-canary: expected input tensors are required when enable_write_input_assert=True"
            )
        expected_input_tokens_host = expected_input_tokens.detach().to(
            device=work_device, dtype=torch.int64
        )
        expected_input_positions_host = expected_input_positions.detach().to(
            device=work_device, dtype=torch.int64
        )
    else:
        if expected_input_tokens is not None or expected_input_positions is not None:
            raise ValueError(
                "kv-canary: expected input tensors must be None when enable_write_input_assert=False"
            )
        expected_input_tokens_host = None
        expected_input_positions_host = None

    violation_rows: list[list[int]] = []
    total_slots_written = 0

View on GitHub (pinned to 0132848349)

Solutions

1. Supply both expected_input_tokens and expected_input_positions
2. Or disable enable_write_input_assert if parity of the assert path is not being tested
3. Keep a single kwargs dict shared between CUDA and reference calls so they stay in sync

Example fix

# before
launch_canary_write_kernel_torch_reference(ctx, plan, ..., enable_write_input_assert=True)
# after
launch_canary_write_kernel_torch_reference(ctx, plan, ..., enable_write_input_assert=True,
    expected_input_tokens=exp_tokens, expected_input_positions=exp_positions)

if enable_write_input_assert:
    assert expected_input_tokens is not None and expected_input_positions is not None

def ref_assert_args_ok(flag: bool, t, p) -> bool:
    return (flag and t is not None and p is not None) or (not flag and t is None and p is None)

When it happens

Trigger: Calling launch_canary_write_kernel_torch_reference(..., enable_write_input_assert=True) with one or both expected tensors None.

Common situations: Mirroring a CUDA-path call in a parity test but forgetting the expected tensors; enabling asserts in a shared helper whose callers pass None defaults.

Understand the failure class

Background: Missing required parameter errors: what 'X is required' and 'the required X param is missing' mean, and how to fix them — this error's family across 27 libraries.

Related errors

• kv-canary: canary_buf slot stride must hold at least 4 int64
• kv-canary: expected input tensors are required when enable_w
• kv-canary: canary_buf slot stride must hold at least 4 int64
• kv-canary: expected input tensors must be None when enable_w
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/47eab941e6e8de03. Report an issue: GitHub.