sgl-project/sglang · error · ValueError

kv-canary: {name} must be contiguous

Error message

kv-canary: {name} must be contiguous

What it means

Several kv-canary launchers call _assert_contiguous on each tensor argument because the Triton kernels compute flat pointer offsets assuming C-contiguous memory. A non-contiguous tensor (e.g. a transposed or strided view) raises ValueError with the tensor's name.

Source

Thrown at python/sglang/kernels/ops/kv_canary/verify.py:38

    """Unique tag per (head | tail | sweep) × (K | V) × (FULL | SWA) launch."""

    HEAD_K_FULL = 0
    HEAD_V_FULL = 1
    TAIL_K_FULL = 2
    TAIL_V_FULL = 3
    SWEEP_K_FULL = 4
    SWEEP_V_FULL = 5
    HEAD_K_SWA = 6
    HEAD_V_SWA = 7
    TAIL_K_SWA = 8
    TAIL_V_SWA = 9
    SWEEP_K_SWA = 10
    SWEEP_V_SWA = 11


def _assert_contiguous(tensor: torch.Tensor, name: str) -> None:
    if not tensor.is_contiguous():
        raise ValueError(f"kv-canary: {name} must be contiguous")


@dataclass(frozen=True, slots=True, kw_only=True)
class RealKvSource:
    """One piece of real KV the canary folds into its fingerprint.

    Slot access invariant (must hold for every source, regardless of underlying layout) — for a given slot_idx,
    the canary reads exactly these bytes:

        tensor[
            slot_idx // page_size,
            (slot_idx % page_size) * num_bytes_per_token
            : ((slot_idx % page_size) + 1) * num_bytes_per_token
        ]

    Note that ``tensor`` may have "holes" in dim 1 — ``tensor.shape[1]`` can exceed ``page_size *
    num_bytes_per_token``. Trailing bytes of each row are ignored by the canary; this is exactly how the
    abstraction accommodates pools whose per-row layout interleaves canary-relevant bytes with other metadata

View on GitHub (pinned to 0132848349)

Solutions

1. Materialize a contiguous copy: t = t.contiguous() before the call
2. Fix the producer to allocate the layout the kernel expects rather than re-striding later
3. Check t.is_contiguous() in debug builds of the caller to catch stray views early

Example fix

# before
launch_verify(..., k_cache=kv[:, :, ::2, :])
# after
k = kv[:, :, ::2, :].contiguous()
launch_verify(..., k_cache=k)

assert t.is_contiguous() for t in inputs  # e.g.
for name, t in [('k', k), ('v', v)]:
    assert t.is_contiguous(), name

def is_contiguous(t: torch.Tensor) -> bool:
    return t.is_contiguous()

When it happens

Trigger: Passing a transposed view (t.t()), a sliced sub-block (t[:, ::2]), or a tensor from .expand() to launch_canary_verify_kernel, launch_canary_write_kernel, or the offsets-kernel input validator.

Common situations: Slicing K/V caches with a stride (e.g. taking every other head); reusing views created for other kernels that tolerate strides.

Related errors

• {fn_name}: dst entry dims (dims {entry_start_dim}..{dst.ndim
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• unified_kv dtype mismatch: kv={unified_kv.dtype}, q={q.dtype
• `mixed_qkv` must be contiguous in the last dim.
• MXFP8 fused prologue requires contiguous interleaved SFK/SFV

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5c5f5684bf25da07. Report an issue: GitHub.