sgl-project/sglang · error · ValueError

kv-canary: {name} must be 2-D, got shape {tuple(tensor.shape

Error message

kv-canary: {name} must be 2-D, got shape {tuple(tensor.shape)}

What it means

The req_to_token mapping must be a 2-D [max_reqs, max_context] tensor; the offsets kernel validates ndim==2 before computing row addresses from stride(0).

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/utils.py:41


def _require_dtype(tensor: torch.Tensor, name: str, dtype: torch.dtype) -> None:
    if tensor.dtype != dtype:
        raise ValueError(
            f"kv-canary: {name} must have dtype {dtype}, got {tensor.dtype}"
        )


def _require_1d(tensor: torch.Tensor, name: str) -> None:
    if tensor.ndim != 1:
        raise ValueError(
            f"kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape)}"
        )


def _require_2d(tensor: torch.Tensor, name: str) -> None:
    if tensor.ndim != 2:
        raise ValueError(
            f"kv-canary: {name} must be 2-D, got shape {tuple(tensor.shape)}"
        )


def _require_len(tensor: torch.Tensor, name: str, expected: int) -> None:
    _require_1d(tensor=tensor, name=name)
    actual = int(tensor.shape[0])
    if actual != expected:
        raise ValueError(f"kv-canary: {name} length must be {expected}, got {actual}")


def _require_min_len(tensor: torch.Tensor, name: str, minimum: int) -> None:
    _require_1d(tensor=tensor, name=name)
    actual = int(tensor.shape[0])
    if actual < minimum:
        raise ValueError(f"kv-canary: {name} length must be >= {minimum}, got {actual}")

View on GitHub (pinned to 0132848349)

Solutions

  1. Reshape req_to_token to 2-D: req_to_token = req_to_token.reshape(max_reqs, max_context_len)
  2. Verify the tensor comes from the KV cache manager's req_to_token pool unchanged

Example fix

// before
req_to_token = req_to_token.reshape(-1)
// after
req_to_token = req_to_token.reshape(max_reqs, max_context_len)
Defensive patterns

Strategy: type-guard

Validate before calling

req_to_token = req_to_token.reshape(max_reqs, max_context_len)

Type guard

def is_2d(t: torch.Tensor) -> bool:
    return t.ndim == 2

Prevention

When it happens

Trigger: Calling launch_plan_offsets_kernel with req_to_token that is 1-D, 3-D, or a flattened view instead of a proper 2-D mapping.

Common situations: The mapping tensor being flattened for transport and not reshaped back; passing a per-shard slice with the wrong rank.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/16bb7623508c36a5. Report an issue: GitHub.