{"record":{"id":"c5844e7570dde6fd","repo":"sgl-project/sglang","slug":"kv-canary-realkvsource-tensor-dim-1-byte-width-mu","errorCode":null,"errorMessage":"kv-canary: RealKvSource.tensor dim-1 byte width must be a multiple of 16, got {row_stride_bytes} bytes (shape={tuple(self.tensor.shape)}, dtype={self.tensor.dtype})","messagePattern":"kv-canary: RealKvSource\\.tensor dim-1 byte width must be a multiple of 16, got (.+?) bytes \\(shape=(.+?), dtype=(.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/verify.py","lineNumber":118,"sourceCode":"                f\"kv-canary: RealKvSource.num_bytes_per_token must be a positive multiple of 16, \"\n                f\"got {self.num_bytes_per_token}\"\n            )\n        if (\n            self.read_bytes <= 0\n            or self.read_bytes > self.num_bytes_per_token\n            or self.read_bytes % 16 != 0\n        ):\n            raise ValueError(\n                f\"kv-canary: RealKvSource.read_bytes must be a positive multiple of 16 in \"\n                f\"(0, num_bytes_per_token={self.num_bytes_per_token}], got {self.read_bytes}\"\n            )\n        if self.tensor.ndim < 2:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.tensor must be at least 2-D, got shape {tuple(self.tensor.shape)}\"\n            )\n        row_stride_bytes = int(self.tensor.shape[1]) * self.tensor.element_size()\n        if row_stride_bytes % 16 != 0:\n            raise ValueError(\n                f\"kv-canary: RealKvSource.tensor dim-1 byte width must be a multiple of 16, \"\n                f\"got {row_stride_bytes} bytes (shape={tuple(self.tensor.shape)}, \"\n                f\"dtype={self.tensor.dtype})\"\n            )\n\n\n@dataclass(frozen=True, slots=True, kw_only=True)\nclass VerifyOrWriteContext:\n    \"\"\"Shared launch context for canary verify/write kernels.\n\n    Fields:\n        canary_buf: Canary buffer this launch verifies or writes, shape [num_slots, slot_stride_bytes], uint8.\n            slot_stride_bytes is read from canary_buf.shape[1].\n        kernel_kind: CanaryLaunchTag identifying which launch fired. Stamped (as int) into every violation row\n            so host can attribute a violation back to its source launch.\n        violation_ring: Global append-only sink, shape [ring_capacity, VIOLATION_FIELDS], int64. Shared across\n            all canary launches; fill-once.\n        violation_write_index: Global monotonic violation counter, shape [1], int32.","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/verify.py#L100-L136","documentation":"Beyond rank, the kernel requires each row of RealKvSource.tensor to be a whole number of 16-byte units: dim-1 byte width (shape[1] * element_size) must be divisible by 16 for the vectorized 16-byte reads. __post_init__ raises ValueError with the computed row width otherwise.","triggerScenarios":"A tensor whose shape[1] * element_size is not a multiple of 16 — e.g. an odd number of fp16 elements per row (shape[1]=7 with 2-byte dtype gives 14 bytes), or a padded layout with a non-16 stride.","commonSituations":"Exotic head counts/head dims; column slicing that leaves a non-16-byte row width; viewing the cache with an incompatible dim-1 size.","solutions":["Pad shape[1] (with zeros) so shape[1] * element_size % 16 == 0","Verify num_bytes_per_token matches shape[1] * element_size and adjust the view/reshape","Avoid slicing columns; pass the full row and limit reads via read_bytes instead"],"exampleFix":"# before\nsrc = RealKvSource(tensor=kv[:, :7], ...)  # 14 bytes/row for fp16\n# after\nsrc = RealKvSource(tensor=kv, ...)  # full rows; use read_bytes to limit bytes read","handlingStrategy":"validation","validationCode":"row = tensor.shape[1] * tensor.element_size()\nassert row % 16 == 0, f\"row width {row} bytes not 16-aligned\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't column-slice the KV tensor; limit reads via read_bytes instead of shape slicing"],"tags":["kv-cache","alignment","strides","validation"],"backgroundTag":"alignment-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}