{"record":{"id":"7c3c0451087babf3","repo":"sgl-project/sglang","slug":"kv-canary-realkvsource-num-bytes-per-token-must-b","errorCode":null,"errorMessage":"kv-canary: RealKvSource.num_bytes_per_token must be a positive multiple of 16, got {self.num_bytes_per_token}","messagePattern":"kv-canary: RealKvSource\\.num_bytes_per_token must be a positive multiple of 16, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/verify.py","lineNumber":99,"sourceCode":"        page_size: Number of slots packed into one row of dim 0. ``>= 1``.\n        num_bytes_per_token: Bytes per slot in the dim-1 strip the canary reads. Must be a positive\n            multiple of 16.\n        read_bytes: Leading bytes (out of ``num_bytes_per_token``) per slot folded into the fingerprint.\n            Must be a positive multiple of 16, ``<= num_bytes_per_token``.\n    \"\"\"\n\n    tensor: torch.Tensor\n    page_size: int\n    num_bytes_per_token: int\n    read_bytes: int\n\n    def __post_init__(self) -> None:\n        if self.page_size < 1:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.page_size must be >= 1, got {self.page_size}\"\n            )\n        if self.num_bytes_per_token <= 0 or self.num_bytes_per_token % 16 != 0:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.num_bytes_per_token must be a positive multiple of 16, \"\n                f\"got {self.num_bytes_per_token}\"\n            )\n        if (\n            self.read_bytes <= 0\n            or self.read_bytes > self.num_bytes_per_token\n            or self.read_bytes % 16 != 0\n        ):\n            raise ValueError(\n                f\"kv-canary: RealKvSource.read_bytes must be a positive multiple of 16 in \"\n                f\"(0, num_bytes_per_token={self.num_bytes_per_token}], got {self.read_bytes}\"\n            )\n        if self.tensor.ndim < 2:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.tensor must be at least 2-D, got shape {tuple(self.tensor.shape)}\"\n            )\n        row_stride_bytes = int(self.tensor.shape[1]) * self.tensor.element_size()\n        if row_stride_bytes % 16 != 0:","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/verify.py#L81-L117","documentation":"RealKvSource.num_bytes_per_token must be a positive multiple of 16 because the canary kernel reads KV rows in 16-byte chunks (vectorized loads). __post_init__ raises ValueError for zero, negative, or non-multiple-of-16 values.","triggerScenarios":"Constructing RealKvSource with num_bytes_per_token derived from head_dim * head_count * element_size that isn't a multiple of 16 (e.g. bf16 with an odd byte count), or passing 0.","commonSituations":"Switching model config to an unusual head_dim/dtype combination; forgetting to multiply by 2 bytes for fp16/bf16 when computing byte width.","solutions":["Compute num_bytes_per_token = num_heads * head_dim * tensor.element_size() and verify % 16 == 0","Round up to the next multiple of 16 if the layout has padding","Check for accidentally passing element count instead of byte count (missing element_size factor)"],"exampleFix":"# before\nsrc = RealKvSource(num_bytes_per_token=num_heads * head_dim, ...)  # forgot bytes\n# after\nsrc = RealKvSource(num_bytes_per_token=num_heads * head_dim * kv.element_size(), ...)","handlingStrategy":"validation","validationCode":"nb = num_heads * head_dim * kv.element_size()\nassert nb > 0 and nb % 16 == 0, nb","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Compute byte widths with element_size(); sanity-check % 16 for new dtype/head_dim combos"],"tags":["kv-cache","alignment","byte-width","validation"],"backgroundTag":"alignment-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}