sgl-project/sglang · error · ValueError

kv-canary: canary_buf slot stride must hold at least 4 int64

Error message

kv-canary: canary_buf slot stride must hold at least 4 int64 fields, got {slot_stride_i64}

What it means

The torch reference implementation of the canary verify kernel views canary_buf as int64 and uses each row's stride as the slot record layout; it needs at least 4 int64 fields per slot (the canary record schema). A narrower buffer means the buffer was allocated with the wrong slot stride.

Source

Thrown at python/sglang/kernels/ops/kv_canary/verify_ref.py:92

        # Skip SGLang's padded-token dummy KV slot so unfilled req_to_token entries (zero-initialized) do not
        # produce spurious chain_hash / position violations.
        if s != consts.TOKEN_TO_KV_SLOT_PADDING:
            kept_slots.append(s)
            kept_expected_positions.append(int(expected_positions_host[k].item()))
            kept_expected_input_ids.append(int(expected_input_ids_host[k].item()))
            kept_prev_slots.append(int(prev_slot_indices_host[k].item()))
    active = len(kept_slots)
    if active <= 0:
        return
    slot_indices_list: list[int] = kept_slots
    expected_positions_list: list[int] = kept_expected_positions
    expected_input_ids_list: list[int] = kept_expected_input_ids
    prev_slot_indices_list: list[int] = kept_prev_slots

    buf_i64 = canary_buf.detach().to(device=work_device).contiguous().view(torch.int64)
    slot_stride_i64 = int(buf_i64.shape[1])
    if slot_stride_i64 < 4:
        raise ValueError(
            f"kv-canary: canary_buf slot stride must hold at least 4 int64 fields, got {slot_stride_i64}"
        )

    violation_rows: list[list[int]] = []

    for k in range(active):
        slot_idx = slot_indices_list[k]
        expected_position = expected_positions_list[k]
        expected_input_id = expected_input_ids_list[k]
        prev_slot = prev_slot_indices_list[k]

        stored_token = int(buf_i64[slot_idx, consts.CANARY_FIELD_TOKEN].item())
        stored_position = int(buf_i64[slot_idx, consts.CANARY_FIELD_POSITION].item())
        stored_chain_hash = int(buf_i64[slot_idx, consts.CANARY_FIELD_PREV_HASH].item())
        stored_real_kv_hash = int(
            buf_i64[slot_idx, consts.CANARY_FIELD_REAL_KV_HASH].item()
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Allocate canary_buf with the canonical slot stride (>= 4 int64 = 32 bytes per slot); prefer the library's buffer allocator helper
2. If you changed the record schema, ensure buf.view(torch.int64).shape[1] >= 4 before calling
3. Add a unit assert on the stride in your test harness

Example fix

# before
canary_buf = torch.empty(num_slots, 16, dtype=torch.uint8, device=dev)  # 2 int64 fields
# after
canary_buf = torch.empty(num_slots, 4 * 8, dtype=torch.uint8, device=dev)  # 4 int64 fields/slot

stride = canary_buf.view(torch.int64).shape[1]
assert stride >= 4, f"canary_buf stride {stride} < 4 int64 fields"

def canary_buf_stride_ok(buf: torch.Tensor) -> bool:
    return buf.view(torch.int64).shape[1] >= 4

When it happens

Trigger: Calling launch_canary_verify_kernel_torch_reference with a canary_buf whose int64 row stride (shape[1] after .view(torch.int64)) is < 4, e.g. a buffer allocated with fewer than 32 bytes per slot.

Common situations: Allocating canary_buf manually with a custom stride instead of via the provided allocator; porting the canary to a different slot schema and forgetting the reference's 4-field minimum.

Related errors

• kv-canary: canary_buf slot stride must hold at least 4 int64
• kv-canary: expected input tensors are required when enable_w
• kv-canary: expected input tensors must be None when enable_w
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i
• kv-canary: launch_canary_plan_kernels verify_capacity={verif

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a389dfbcb5d17f6b. Report an issue: GitHub.