sgl-project/sglang · error · ValueError

kv-canary: read_bytes must be a multiple of {_REAL_KV_READ_A

Error message

kv-canary: read_bytes must be a multiple of {_REAL_KV_READ_ALIGN}, got {requested}

What it means

kv-canary requires read_bytes to be aligned to _REAL_KV_READ_ALIGN because it reads the real KV pool with aligned vector loads. A non-multiple size would produce misaligned reads, so _clip_read_bytes_aligned raises ValueError.

Source

Thrown at python/sglang/srt/kv_canary/pool_patcher/buffer_alloc.py:58

    """
    if num_bytes_per_token <= 0 or num_bytes_per_token % _REAL_KV_READ_ALIGN != 0:
        raise ValueError(
            "kv-canary: num_bytes_per_token must be a positive multiple of "
            f"{_REAL_KV_READ_ALIGN}, got {num_bytes_per_token}"
        )
    if requested == 0:
        return 0
    if requested == sys.maxsize:
        return num_bytes_per_token
    if requested < 0:
        raise ValueError(f"kv-canary: read_bytes must be non-negative, got {requested}")
    if requested > num_bytes_per_token:
        raise ValueError(
            "kv-canary: read_bytes must be <= num_bytes_per_token "
            f"({num_bytes_per_token}), got {requested}"
        )
    if requested % _REAL_KV_READ_ALIGN != 0:
        raise ValueError(
            "kv-canary: read_bytes must be a multiple of "
            f"{_REAL_KV_READ_ALIGN}, got {requested}"
        )
    return requested


def make_row_source(
    *,
    layer_buffer: torch.Tensor,
    read_bytes: int,
) -> Tuple[RealKvSource, ...]:
    contiguous = layer_buffer.contiguous()
    num_slots = int(contiguous.shape[0])
    if num_slots == 0 or read_bytes == 0:
        return ()
    flat = contiguous.view(torch.uint8).reshape(num_slots, -1)
    num_bytes_per_token = int(flat.shape[1])
    clipped = _clip_read_bytes_aligned(

View on GitHub (pinned to 0132848349)

Solutions

Round down to the nearest multiple of the alignment: read_bytes - (read_bytes % _REAL_KV_READ_ALIGN)
Use sys.maxsize (resolves to num_bytes_per_token, which is aligned) or 0
Import _REAL_KV_READ_ALIGN from the module and compute aligned sizes explicitly

Example fix

// before
src = make_row_source(pool, read_bytes=100)
// after
from sglang.srt.kv_canary.pool_patcher.buffer_alloc import _REAL_KV_READ_ALIGN
aligned = (100 // _REAL_KV_READ_ALIGN) * _REAL_KV_READ_ALIGN
src = make_row_source(pool, read_bytes=aligned)

Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.kv_canary.pool_patcher.buffer_alloc import _REAL_KV_READ_ALIGN
read_bytes = read_bytes - (read_bytes % _REAL_KV_READ_ALIGN)

Type guard

def is_aligned(n: int, align: int = _REAL_KV_READ_ALIGN) -> bool:
    return n % align == 0

Prevention

Round sizes down to the alignment constant before calling
Prefer sys.maxsize or 0 which are always valid

When it happens

Trigger: Calling make_row_source or make_packed_source with a read_bytes not divisible by the alignment constant (e.g. 100 when alignment is 16).

Common situations: Choosing a round decimal size like 100 or 504; deriving half/third sizes that break alignment (65 // 2 = 32 is fine but 66 // 2 = 33 is not for align 16).

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bd779440e1946371. Report an issue: GitHub.