sgl-project/sglang · error · ValueError

kv-canary: canary_buf slot stride must hold at least 4 int64

Error message

kv-canary: canary_buf slot stride must hold at least 4 int64 fields, got {slot_stride_i64}

What it means

The torch reference write kernel views canary_buf as int64 rows and requires at least 4 int64 fields per slot (the canary record schema). A smaller row stride means the buffer layout does not match what the write reference (and the CUDA kernel it must match byte-for-byte) expects.

Source

Thrown at python/sglang/kernels/ops/kv_canary/write_ref.py:76

    positions_host = positions.detach().to(device=work_device, dtype=torch.int64)
    out_cache_loc_host = out_cache_loc.detach().to(
        device=work_device, dtype=torch.int64
    )

    total_entries = int(write_offsets_host[active_reqs].item())
    if total_entries <= 0:
        return

    buf_i64 = (
        canary_buf.detach()
        .to(device=work_device)
        .contiguous()
        .view(torch.int64)
        .clone()
    )
    slot_stride_i64 = int(buf_i64.shape[1])
    if slot_stride_i64 < 4:
        raise ValueError(
            f"kv-canary: canary_buf slot stride must hold at least 4 int64 fields, got {slot_stride_i64}"
        )

    if enable_write_input_assert:
        if expected_input_tokens is None or expected_input_positions is None:
            raise ValueError(
                "kv-canary: expected input tensors are required when enable_write_input_assert=True"
            )
        expected_input_tokens_host = expected_input_tokens.detach().to(
            device=work_device, dtype=torch.int64
        )
        expected_input_positions_host = expected_input_positions.detach().to(
            device=work_device, dtype=torch.int64
        )
    else:
        if expected_input_tokens is not None or expected_input_positions is not None:
            raise ValueError(
                "kv-canary: expected input tensors must be None when enable_write_input_assert=False"

View on GitHub (pinned to 0132848349)

Solutions

1. Allocate canary_buf with >= 4 int64 fields (32 bytes) per slot, ideally via the library allocator
2. Assert buf.view(torch.int64).shape[1] >= 4 in your test setup
3. Keep verify and write buffers allocated from the same helper so strides stay consistent

Example fix

# before
canary_buf = torch.empty(num_slots, 2 * 8, dtype=torch.uint8, device=dev)
# after
canary_buf = torch.empty(num_slots, 4 * 8, dtype=torch.uint8, device=dev)

assert canary_buf.view(torch.int64).shape[1] >= 4

def canary_buf_stride_ok(buf: torch.Tensor) -> bool:
    return buf.view(torch.int64).shape[1] >= 4

When it happens

Trigger: Calling launch_canary_write_kernel_torch_reference with a canary_buf whose per-slot byte stride is < 32 bytes (< 4 int64 fields).

Common situations: Hand-allocating canary_buf with a custom/narrower stride; schema changes to the canary record without updating all allocators.

Related errors

• kv-canary: canary_buf slot stride must hold at least 4 int64
• kv-canary: expected input tensors are required when enable_w
• kv-canary: expected input tensors must be None when enable_w
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i
• kv-canary: launch_canary_plan_kernels verify_capacity={verif

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4747f41275a65237. Report an issue: GitHub.