sgl-project/sglang · error · ValueError

kv-canary: {name} length must be {expected}, got {actual}

Error message

kv-canary: {name} length must be {expected}, got {actual}

What it means

Per-request input vectors (req_pool_indices, prefix_lens, extend_seq_lens, etc.) must have length exactly equal to the declared batch size bs; the kernel reads one element per request with no bounds slack.

Source

Thrown at python/sglang/kernels/ops/kv_canary/plan/utils.py:50

def _require_1d(tensor: torch.Tensor, name: str) -> None:
    if tensor.ndim != 1:
        raise ValueError(
            f"kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape)}"
        )


def _require_2d(tensor: torch.Tensor, name: str) -> None:
    if tensor.ndim != 2:
        raise ValueError(
            f"kv-canary: {name} must be 2-D, got shape {tuple(tensor.shape)}"
        )


def _require_len(tensor: torch.Tensor, name: str, expected: int) -> None:
    _require_1d(tensor=tensor, name=name)
    actual = int(tensor.shape[0])
    if actual != expected:
        raise ValueError(f"kv-canary: {name} length must be {expected}, got {actual}")


def _require_min_len(tensor: torch.Tensor, name: str, minimum: int) -> None:
    _require_1d(tensor=tensor, name=name)
    actual = int(tensor.shape[0])
    if actual < minimum:
        raise ValueError(f"kv-canary: {name} length must be >= {minimum}, got {actual}")


def _require_same_device(
    reference: torch.Tensor,
    reference_name: str,
    tensors: tuple[tuple[torch.Tensor, str], ...],
) -> None:
    for tensor, name in tensors:
        if tensor.device != reference.device:
            raise ValueError(
                f"kv-canary: {name} must be on {reference_name}'s device "

View on GitHub (pinned to 0132848349)

Solutions

  1. Recompute all per-request tensors and bs from the same request list so lengths agree by construction
  2. Print each tensor's shape[0] vs bs to find the diverging input named in the message

Example fix

// before
bs = len(all_reqs)
lens = lens_for_active_only  # shorter than all_reqs
// after
active = [r for r in all_reqs if not r.finished]
bs = len(active)
lens = build_lens(active)
Defensive patterns

Strategy: validation

Validate before calling

assert all(t.shape[0] == bs for t in (req_pool_indices, prefix_lens, extend_seq_lens))

Prevention

When it happens

Trigger: Calling launch_plan_offsets_kernel where a per-request input's shape[0] != bs — e.g. tensors sliced to a subset of the batch, or bs computed from a different list than the tensors.

Common situations: Batch filtering (removing finished requests) applied to bs but not the tensors, or vice versa; off-by-one in slicing during chunked prefill.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/66f3cfd2e3556352. Report an issue: GitHub.