sgl-project/sglang · error · ValueError
kv-canary: must be 1-D, got shape
Error message
kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape)} What it means
kv-canary plan validation: the named tensor must be 1-D. Reshape it to a flat vector matching the expected plan length.
Solutions
- Reshape the named tensor to 1-D: t = t.reshape(-1) or t.squeeze(-1) as appropriate
- Check the error's tensor name and shape to see which dimension is spurious
Example fix
// before prefix_lens = prefix_lens.unsqueeze(-1) # [bs, 1] launch_plan_offsets_kernel(..., prefix_lens=prefix_lens, ...) // after prefix_lens = prefix_lens.reshape(-1) # [bs] launch_plan_offsets_kernel(..., prefix_lens=prefix_lens, ...)
Defensive patterns
Strategy: type-guard
Validate before calling
t = t.reshape(-1)
Type guard
def is_1d(t: torch.Tensor) -> bool:
return t.ndim == 1 Prevention
- Normalize per-request inputs with reshape(-1) before launch
When it happens
Trigger: Calling launch_plan_offsets_kernel with e.g. prefix_lens of shape [bs, 1] or [1, bs] instead of [bs]; passing a scalar where a length-1 vector is required.
Common situations: Inputs coming from code that adds a trailing dimension for other kernels; squeeze/unsqueeze mismatches after a refactor.
Understand the failure class
Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.
Related errors
- kv-canary: must be 2-D, got shape
- chunk_size should be a int, or a tuple of length 2 or 3…
- kv-canary: cuda_graph_max_bs must be non-negative, got
- kv-canary: max_prefill_tokens must be positive, got
- kv-canary: max_seq_len_per_req must be positive, got
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/fda5137b1fea3141.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/kernels/ops/kv_canary/plan/utils.py:34
when the caller passes ``None`` we substitute a one-element sentinel tensor and set ``lut_len=0``;
the kernel's constexpr branch guarantees no dereference happens. Dtype matches the production LUT
(int64) so Triton ``tl.load`` element typing stays consistent.
"""
if lut is not None:
return lut, int(lut.shape[0]), True
return torch.zeros(1, dtype=torch.int64, device=device), 0, False
def _require_dtype(tensor: torch.Tensor, name: str, dtype: torch.dtype) -> None:
if tensor.dtype != dtype:
raise ValueError(
f"kv-canary: {name} must have dtype {dtype}, got {tensor.dtype}"
)
def _require_1d(tensor: torch.Tensor, name: str) -> None:
if tensor.ndim != 1:
raise ValueError(
f"kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape)}"
)
def _require_2d(tensor: torch.Tensor, name: str) -> None:
if tensor.ndim != 2:
raise ValueError(
f"kv-canary: {name} must be 2-D, got shape {tuple(tensor.shape)}"
)
def _require_len(tensor: torch.Tensor, name: str, expected: int) -> None:
_require_1d(tensor=tensor, name=name)
actual = int(tensor.shape[0])
if actual != expected:
raise ValueError(f"kv-canary: {name} length must be {expected}, got {actual}")
View on GitHub (pinned to 0132848349)