sgl-project/sglang · error · ValueError

kv-canary: RealKvSource.tensor must be at least 2-D, got sha

Error message

kv-canary: RealKvSource.tensor must be at least 2-D, got shape {tuple(self.tensor.shape)}

What it means

The RealKvSource tensor holding the actual KV data must be at least 2-D (rows = tokens/pages, dim 1 = per-token bytes); a 1-D or 0-D tensor raises ValueError in __post_init__ because the kernel indexes rows by token.

Source

Thrown at python/sglang/kernels/ops/kv_canary/verify.py:113

            raise ValueError(
                f"kv-canary: RealKvSource.page_size must be >= 1, got {self.page_size}"
            )
        if self.num_bytes_per_token <= 0 or self.num_bytes_per_token % 16 != 0:
            raise ValueError(
                f"kv-canary: RealKvSource.num_bytes_per_token must be a positive multiple of 16, "
                f"got {self.num_bytes_per_token}"
            )
        if (
            self.read_bytes <= 0
            or self.read_bytes > self.num_bytes_per_token
            or self.read_bytes % 16 != 0
        ):
            raise ValueError(
                f"kv-canary: RealKvSource.read_bytes must be a positive multiple of 16 in "
                f"(0, num_bytes_per_token={self.num_bytes_per_token}], got {self.read_bytes}"
            )
        if self.tensor.ndim < 2:
            raise ValueError(
                f"kv-canary: RealKvSource.tensor must be at least 2-D, got shape {tuple(self.tensor.shape)}"
            )
        row_stride_bytes = int(self.tensor.shape[1]) * self.tensor.element_size()
        if row_stride_bytes % 16 != 0:
            raise ValueError(
                f"kv-canary: RealKvSource.tensor dim-1 byte width must be a multiple of 16, "
                f"got {row_stride_bytes} bytes (shape={tuple(self.tensor.shape)}, "
                f"dtype={self.tensor.dtype})"
            )


@dataclass(frozen=True, slots=True, kw_only=True)
class VerifyOrWriteContext:
    """Shared launch context for canary verify/write kernels.

    Fields:
        canary_buf: Canary buffer this launch verifies or writes, shape [num_slots, slot_stride_bytes], uint8.
            slot_stride_bytes is read from canary_buf.shape[1].

View on GitHub (pinned to 0132848349)

Solutions

1. Reshape to [num_tokens, bytes_per_token]: kv.view(-1, num_bytes_per_token)
2. If passing a single page, use shape [page_size, num_bytes_per_token]

Example fix

# before
src = RealKvSource(tensor=flat_buffer, ...)  # 1-D
# after
src = RealKvSource(tensor=flat_buffer.view(-1, bytes_per_token), ...)

assert tensor.ndim >= 2, tensor.shape

def is_at_least_2d(t: torch.Tensor) -> bool:
    return t.ndim >= 2

When it happens

Trigger: Passing a flattened 1-D view of the KV cache, or a scalar-like tensor, as RealKvSource.tensor.

Common situations: Flattening the cache for transport and forgetting to restore [tokens, bytes] shape; passing a single page's raw buffer without reshaping to [page_size, bytes_per_token].

Related errors

• kv-canary: scatter_req_token_ids flat_in must be 1-D, got sh
• kv-canary: scatter_req_token_ids offsets must be 1-D, got sh
• kv-canary: scatter_req_token_ids req_pool_indices must be 1-
• kv-canary: scatter_req_token_ids pool_out must be 2-D, got s
• rope_pool_fused expects pool tensors to be 3-D

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a304ec59a2bba9c3. Report an issue: GitHub.