{"record":{"id":"0d51205eee1bdaa9","repo":"sgl-project/sglang","slug":"kv-canary-realkvsource-read-bytes-must-be-a-posit","errorCode":null,"errorMessage":"kv-canary: RealKvSource.read_bytes must be a positive multiple of 16 in (0, num_bytes_per_token={self.num_bytes_per_token}], got {self.read_bytes}","messagePattern":"kv-canary: RealKvSource\\.read_bytes must be a positive multiple of 16 in \\(0, num_bytes_per_token=(.+?)\\], got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/verify.py","lineNumber":108,"sourceCode":"    num_bytes_per_token: int\n    read_bytes: int\n\n    def __post_init__(self) -> None:\n        if self.page_size < 1:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.page_size must be >= 1, got {self.page_size}\"\n            )\n        if self.num_bytes_per_token <= 0 or self.num_bytes_per_token % 16 != 0:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.num_bytes_per_token must be a positive multiple of 16, \"\n                f\"got {self.num_bytes_per_token}\"\n            )\n        if (\n            self.read_bytes <= 0\n            or self.read_bytes > self.num_bytes_per_token\n            or self.read_bytes % 16 != 0\n        ):\n            raise ValueError(\n                f\"kv-canary: RealKvSource.read_bytes must be a positive multiple of 16 in \"\n                f\"(0, num_bytes_per_token={self.num_bytes_per_token}], got {self.read_bytes}\"\n            )\n        if self.tensor.ndim < 2:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.tensor must be at least 2-D, got shape {tuple(self.tensor.shape)}\"\n            )\n        row_stride_bytes = int(self.tensor.shape[1]) * self.tensor.element_size()\n        if row_stride_bytes % 16 != 0:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.tensor dim-1 byte width must be a multiple of 16, \"\n                f\"got {row_stride_bytes} bytes (shape={tuple(self.tensor.shape)}, \"\n                f\"dtype={self.tensor.dtype})\"\n            )\n\n\n@dataclass(frozen=True, slots=True, kw_only=True)\nclass VerifyOrWriteContext:","sourceCodeStart":90,"sourceCodeEnd":126,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/verify.py#L90-L126","documentation":"RealKvSource.read_bytes (how many bytes per token the canary actually fingerprints) must be a positive multiple of 16 and at most num_bytes_per_token. __post_init__ raises ValueError otherwise, since the kernel reads in 16-byte units and cannot read past a row.","triggerScenarios":"Setting read_bytes=0 (fingerprint nothing), read_bytes > num_bytes_per_token, or a non-multiple-of-16 sampling width when constructing the source.","commonSituations":"Tuning canary cost by sampling fewer bytes per token and choosing e.g. 100 instead of 96; copying read_bytes from a config expressed in elements not bytes.","solutions":["Pick a multiple of 16 within [16, num_bytes_per_token], e.g. min(64, num_bytes_per_token)","Derive read_bytes from num_bytes_per_token: read_bytes = num_bytes_per_token if num_bytes_per_token <= 64 else ((num_bytes_per_token // 4 + 15) // 16) * 16"],"exampleFix":"# before\nsrc = RealKvSource(..., read_bytes=100)\n# after\nsrc = RealKvSource(..., read_bytes=96)","handlingStrategy":"validation","validationCode":"assert 0 < read_bytes <= num_bytes_per_token and read_bytes % 16 == 0, read_bytes","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Choose read_bytes as a multiple of 16; express it in bytes not elements"],"tags":["kv-cache","alignment","validation","sampling"],"backgroundTag":"alignment-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}