sgl-project/sglang · error · ValueError

kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape

Error message

kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape)}

What it means

Plan-kernel helpers enforce that indexed inputs are 1-D vectors; a 2-D tensor (or scalar/0-D) means the caller reshaped or batched the input in a way the kernel cannot interpret.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/utils.py:34

    when the caller passes ``None`` we substitute a one-element sentinel tensor and set ``lut_len=0``;
    the kernel's constexpr branch guarantees no dereference happens. Dtype matches the production LUT
    (int64) so Triton ``tl.load`` element typing stays consistent.
    """
    if lut is not None:
        return lut, int(lut.shape[0]), True
    return torch.zeros(1, dtype=torch.int64, device=device), 0, False


def _require_dtype(tensor: torch.Tensor, name: str, dtype: torch.dtype) -> None:
    if tensor.dtype != dtype:
        raise ValueError(
            f"kv-canary: {name} must have dtype {dtype}, got {tensor.dtype}"
        )


def _require_1d(tensor: torch.Tensor, name: str) -> None:
    if tensor.ndim != 1:
        raise ValueError(
            f"kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape)}"
        )


def _require_2d(tensor: torch.Tensor, name: str) -> None:
    if tensor.ndim != 2:
        raise ValueError(
            f"kv-canary: {name} must be 2-D, got shape {tuple(tensor.shape)}"
        )


def _require_len(tensor: torch.Tensor, name: str, expected: int) -> None:
    _require_1d(tensor=tensor, name=name)
    actual = int(tensor.shape[0])
    if actual != expected:
        raise ValueError(f"kv-canary: {name} length must be {expected}, got {actual}")

View on GitHub (pinned to 0132848349)

Solutions

  1. Reshape the named tensor to 1-D: t = t.reshape(-1) or t.squeeze(-1) as appropriate
  2. Check the error's tensor name and shape to see which dimension is spurious

Example fix

// before
prefix_lens = prefix_lens.unsqueeze(-1)  # [bs, 1]
launch_plan_offsets_kernel(..., prefix_lens=prefix_lens, ...)
// after
prefix_lens = prefix_lens.reshape(-1)  # [bs]
launch_plan_offsets_kernel(..., prefix_lens=prefix_lens, ...)
Defensive patterns

Strategy: type-guard

Validate before calling

t = t.reshape(-1)

Type guard

def is_1d(t: torch.Tensor) -> bool:
    return t.ndim == 1

Prevention

When it happens

Trigger: Calling launch_plan_offsets_kernel with e.g. prefix_lens of shape [bs, 1] or [1, bs] instead of [bs]; passing a scalar where a length-1 vector is required.

Common situations: Inputs coming from code that adds a trailing dimension for other kernels; squeeze/unsqueeze mismatches after a refactor.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fda5137b1fea3141. Report an issue: GitHub.