{"record":{"id":"bd779440e1946371","repo":"sgl-project/sglang","slug":"kv-canary-read-bytes-must-be-a-multiple-of-real","errorCode":null,"errorMessage":"kv-canary: read_bytes must be a multiple of {_REAL_KV_READ_ALIGN}, got {requested}","messagePattern":"kv-canary: read_bytes must be a multiple of (.+?), got (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/kv_canary/pool_patcher/buffer_alloc.py","lineNumber":58,"sourceCode":"    \"\"\"\n    if num_bytes_per_token <= 0 or num_bytes_per_token % _REAL_KV_READ_ALIGN != 0:\n        raise ValueError(\n            \"kv-canary: num_bytes_per_token must be a positive multiple of \"\n            f\"{_REAL_KV_READ_ALIGN}, got {num_bytes_per_token}\"\n        )\n    if requested == 0:\n        return 0\n    if requested == sys.maxsize:\n        return num_bytes_per_token\n    if requested < 0:\n        raise ValueError(f\"kv-canary: read_bytes must be non-negative, got {requested}\")\n    if requested > num_bytes_per_token:\n        raise ValueError(\n            \"kv-canary: read_bytes must be <= num_bytes_per_token \"\n            f\"({num_bytes_per_token}), got {requested}\"\n        )\n    if requested % _REAL_KV_READ_ALIGN != 0:\n        raise ValueError(\n            \"kv-canary: read_bytes must be a multiple of \"\n            f\"{_REAL_KV_READ_ALIGN}, got {requested}\"\n        )\n    return requested\n\n\ndef make_row_source(\n    *,\n    layer_buffer: torch.Tensor,\n    read_bytes: int,\n) -> Tuple[RealKvSource, ...]:\n    contiguous = layer_buffer.contiguous()\n    num_slots = int(contiguous.shape[0])\n    if num_slots == 0 or read_bytes == 0:\n        return ()\n    flat = contiguous.view(torch.uint8).reshape(num_slots, -1)\n    num_bytes_per_token = int(flat.shape[1])\n    clipped = _clip_read_bytes_aligned(","sourceCodeStart":40,"sourceCodeEnd":76,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/kv_canary/pool_patcher/buffer_alloc.py#L40-L76","documentation":"kv-canary requires read_bytes to be aligned to _REAL_KV_READ_ALIGN because it reads the real KV pool with aligned vector loads. A non-multiple size would produce misaligned reads, so _clip_read_bytes_aligned raises ValueError.","triggerScenarios":"Calling make_row_source or make_packed_source with a read_bytes not divisible by the alignment constant (e.g. 100 when alignment is 16).","commonSituations":"Choosing a round decimal size like 100 or 504; deriving half/third sizes that break alignment (65 // 2 = 32 is fine but 66 // 2 = 33 is not for align 16).","solutions":["Round down to the nearest multiple of the alignment: read_bytes - (read_bytes % _REAL_KV_READ_ALIGN)","Use sys.maxsize (resolves to num_bytes_per_token, which is aligned) or 0","Import _REAL_KV_READ_ALIGN from the module and compute aligned sizes explicitly"],"exampleFix":"// before\nsrc = make_row_source(pool, read_bytes=100)\n// after\nfrom sglang.srt.kv_canary.pool_patcher.buffer_alloc import _REAL_KV_READ_ALIGN\naligned = (100 // _REAL_KV_READ_ALIGN) * _REAL_KV_READ_ALIGN\nsrc = make_row_source(pool, read_bytes=aligned)","handlingStrategy":"validation","validationCode":"from sglang.srt.kv_canary.pool_patcher.buffer_alloc import _REAL_KV_READ_ALIGN\nread_bytes = read_bytes - (read_bytes % _REAL_KV_READ_ALIGN)","typeGuard":"def is_aligned(n: int, align: int = _REAL_KV_READ_ALIGN) -> bool:\n    return n % align == 0","tryCatchPattern":null,"preventionTips":["Round sizes down to the alignment constant before calling","Prefer sys.maxsize or 0 which are always valid"],"tags":["kv-canary","alignment","validation","valueerror"],"backgroundTag":"alignment-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}