sgl-project/sglang · error · ValueError

kv-canary: expected input tensors must be None when enable_w

Error message

kv-canary: expected input tensors must be None when enable_write_input_assert=False

What it means

The mutually-exclusive counterpart of the assert-mode check: with enable_write_input_assert=False the kernel has no path that consumes expected tensors, so passing either expected_input_tokens or expected_input_positions is rejected to catch inconsistent configuration.

Source

Thrown at python/sglang/kernels/ops/kv_canary/write.py:210

        )

    _assert_contiguous(canary_buf, "canary_buf")
    _assert_contiguous(plan.write_offsets, "plan.write_offsets")
    _assert_contiguous(plan.write_seed_slot_indices, "plan.write_seed_slot_indices")
    _assert_contiguous(plan.write_num_valid_reqs, "plan.write_num_valid_reqs")
    _assert_contiguous(input_ids, "input_ids")
    _assert_contiguous(positions, "positions")
    _assert_contiguous(out_cache_loc, "out_cache_loc")
    if enable_write_input_assert:
        if expected_input_tokens is None or expected_input_positions is None:
            raise ValueError(
                "kv-canary: expected input tensors are required when enable_write_input_assert=True"
            )
        _assert_contiguous(expected_input_tokens, "expected_input_tokens")
        _assert_contiguous(expected_input_positions, "expected_input_positions")
    else:
        if expected_input_tokens is not None or expected_input_positions is not None:
            raise ValueError(
                "kv-canary: expected input tensors must be None when enable_write_input_assert=False"
            )
    _assert_contiguous(context.violation_ring, "violation_ring")
    _assert_contiguous(context.violation_write_index, "violation_write_index")
    _assert_contiguous(context.slot_run_counter, "slot_run_counter")
    _assert_contiguous(context.kernel_run_counter, "kernel_run_counter")
    _assert_contiguous(
        context.enable_chain_position_assert, "enable_chain_position_assert"
    )

    padded_bufs, source_params = _build_real_kv_source_abi(
        real_kv_sources=real_kv_sources, device=canary_buf.device
    )

    module = _jit_canary_write_module()
    module.canary_write_step_cuda(
        canary_buf,
        plan.write_offsets,

View on GitHub (pinned to 0132848349)

Solutions

1. Set enable_write_input_assert=True if you actually want the cross-check
2. Otherwise drop the expected_* arguments from the call
3. Make the flag and the tensors derive from a single config branch so they cannot disagree

Example fix

# before
launch_canary_write_kernel(ctx, plan, ..., enable_write_input_assert=False, expected_input_tokens=t)
# after
launch_canary_write_kernel(ctx, plan, ..., enable_write_input_assert=False)

if not enable_write_input_assert:
    expected_input_tokens = expected_input_positions = None

def assert_inputs_consistent(flag: bool, t, p) -> bool:
    return (flag and t is not None and p is not None) or (not flag and t is None and p is None)

When it happens

Trigger: Calling launch_canary_write_kernel(..., enable_write_input_assert=False, expected_input_tokens=t) (or with expected_input_positions) — tensors supplied while asserts are disabled.

Common situations: Toggling the assert flag off (for perf) while leaving the expected tensors wired in from a debug run; copy-pasted call sites with a stale flag.

Related errors

• kv-canary: launch_canary_plan_kernels requires full_to_swa_i
• kv-canary: launch_canary_plan_kernels verify_capacity={verif
• kv-canary: launch_plan_entries_kernel requires req_to_verify
• kv-canary: write_offsets_len must be positive, got {write_of
• kv-canary: write_req_capacity must be non-negative, got {wri

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8b6916bba1d40f88. Report an issue: GitHub.