sgl-project/sglang · error · ValueError

kv-canary: read_bytes must be non-negative, got {requested}

Error message

kv-canary: read_bytes must be non-negative, got {requested}

What it means

kv-canary's buffer allocation helper rejects negative read_bytes values. _clip_read_bytes_aligned validates the requested per-token read size before allocating canary buffers, and a negative value would corrupt size math, so it fails fast with ValueError.

Source

Thrown at python/sglang/srt/kv_canary/pool_patcher/buffer_alloc.py:51


def _clip_read_bytes_aligned(*, requested: int, num_bytes_per_token: int) -> int:
    """Validate and clip read_bytes for the CUDA fold kernel's 128-bit aligned loads.

    Normalizes sentinels (``sys.maxsize`` -> ``num_bytes_per_token``, ``0`` -> ``0``) and
    rejects negative / unaligned / oversized requests.
    """
    if num_bytes_per_token <= 0 or num_bytes_per_token % _REAL_KV_READ_ALIGN != 0:
        raise ValueError(
            "kv-canary: num_bytes_per_token must be a positive multiple of "
            f"{_REAL_KV_READ_ALIGN}, got {num_bytes_per_token}"
        )
    if requested == 0:
        return 0
    if requested == sys.maxsize:
        return num_bytes_per_token
    if requested < 0:
        raise ValueError(f"kv-canary: read_bytes must be non-negative, got {requested}")
    if requested > num_bytes_per_token:
        raise ValueError(
            "kv-canary: read_bytes must be <= num_bytes_per_token "
            f"({num_bytes_per_token}), got {requested}"
        )
    if requested % _REAL_KV_READ_ALIGN != 0:
        raise ValueError(
            "kv-canary: read_bytes must be a multiple of "
            f"{_REAL_KV_READ_ALIGN}, got {requested}"
        )
    return requested


def make_row_source(
    *,
    layer_buffer: torch.Tensor,
    read_bytes: int,
) -> Tuple[RealKvSource, ...]:

View on GitHub (pinned to 0132848349)

Solutions

Fix the caller to pass a non-negative read_bytes (0 means read nothing, sys.maxsize means read the full per-token size)
If read_bytes is computed, clamp it: max(0, computed) or guard against underflow before calling make_row_source/make_packed_source
Use sys.maxsize instead of -1 when you want the default full-size read

Example fix

// before
src = make_row_source(pool, read_bytes=num_bytes_per_token - overshoot)
// after
src = make_row_source(pool, read_bytes=max(0, num_bytes_per_token - overshoot))

Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.kv_canary.pool_patcher.buffer_alloc import _clip_read_bytes_aligned
if read_bytes < 0:
    raise ValueError('read_bytes must be >= 0')
src = make_row_source(pool, read_bytes=read_bytes)

Type guard

def is_valid_read_bytes(n: int) -> bool:
    return isinstance(n, int) and 0 <= n <= sys.maxsize

Try / catch

try:
    clipped = _clip_read_bytes_aligned(requested=n, num_bytes_per_token=cell)
except ValueError as e:
    logger.warning('bad read_bytes: %s', e); clipped = 0

Prevention

Never pass -1 as a sentinel; use sys.maxsize for 'full size'
Clamp computed read_bytes with max(0, ...)

When it happens

Trigger: Calling make_row_source or make_packed_source with a negative read_bytes argument; e.g. read_bytes=-16 or a computed value that underflows (subtraction yielding a negative number).

Common situations: Computing read_bytes as num_bytes_per_token - something and passing it unclamped; passing -1 as an 'unset' sentinel instead of sys.maxsize (which is the supported 'use full size' sentinel here).

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b2c40fbf72812d35. Report an issue: GitHub.