{"record":{"id":"13afa1628c9d95ab","repo":"sgl-project/sglang","slug":"kv-canary-real-kv-sources-i-tensor-viewed-as","errorCode":null,"errorMessage":"kv-canary: real_kv_sources[{i}].tensor (viewed as uint8) must be 2-D, got {source_u8.dim()}-D","messagePattern":"kv-canary: real_kv_sources\\[(.+?)\\]\\.tensor \\(viewed as uint8\\) must be 2-D, got (.+?)-D","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/verify.py","lineNumber":386,"sourceCode":"\n\ndef _build_real_kv_source_abi(\n    *,\n    real_kv_sources: tuple[RealKvSource, ...],\n    device: torch.device,\n) -> tuple[list[torch.Tensor], torch.Tensor]:\n    padded_bufs: list[torch.Tensor] = []\n    params = torch.zeros(\n        (consts.MAX_REAL_KV_SOURCES, consts.REAL_KV_SOURCE_FIELDS_PER_ENTRY),\n        dtype=torch.int32,\n        device=\"cpu\",\n    )\n\n    for i, source in enumerate(real_kv_sources):\n        _assert_contiguous(source.tensor, f\"real_kv_sources[{i}].tensor\")\n        source_u8 = source.tensor.view(torch.uint8)\n        if source_u8.dim() != 2:\n            raise ValueError(\n                f\"kv-canary: real_kv_sources[{i}].tensor (viewed as uint8) must be 2-D, \"\n                f\"got {source_u8.dim()}-D\"\n            )\n        padded_bufs.append(source_u8)\n        params[i, consts.REAL_KV_SOURCE_FIELD_PAGE_SIZE] = source.page_size\n        params[i, consts.REAL_KV_SOURCE_FIELD_NUM_BYTES_PER_TOKEN] = (\n            source.num_bytes_per_token\n        )\n        params[i, consts.REAL_KV_SOURCE_FIELD_READ_BYTES] = source.read_bytes\n\n    # Pad bufs (never read by the kernel — num_sources bounds the iteration); params already zero.\n    dummy = torch.empty((1, 1), dtype=torch.uint8, device=device)\n    for _ in range(len(real_kv_sources), consts.MAX_REAL_KV_SOURCES):\n        padded_bufs.append(dummy)\n\n    return padded_bufs, params\n","sourceCodeStart":368,"sourceCodeEnd":403,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/verify.py#L368-L403","documentation":"Each RealKvSource.tensor is reinterpreted as a uint8 byte buffer and must be a 2-D [slots, bytes_per_slot] layout because the CUDA ABI writes per-row offsets against a 2-D stride. A 1-D flat buffer or a 3-D tensor cannot be described by the single row-stride field in the ABI params.","triggerScenarios":"Passing a RealKvSource whose .tensor is 1-D (flattened bytes) or 3-D+ (e.g. [num_pages, page_size, head_dim]) to launch_canary_verify_kernel or launch_canary_write_kernel.","commonSituations":"Feeding a raw flat cache allocation, or an unflattened [pages, tokens, dim] KV tensor from a paged cache, directly as a source without reshaping to [slots, slot_bytes].","solutions":["Reshape the tensor to 2-D before passing: tensor.reshape(num_slots, bytes_per_slot) (with .contiguous() so the uint8 view is valid)","If the source is [pages, tokens, dim], flatten the trailing dims: t.view(t.size(0), -1)","Ensure the last dim's stride in bytes matches the page/slot size expected by the kernel params (page_size, num_bytes_per_token)"],"exampleFix":"# before\nsrc = RealKvSource(tensor=flat_bytes_1d, page_size=16, num_bytes_per_token=...)\n# after\nsrc = RealKvSource(tensor=flat_bytes_1d.view(num_slots, slot_bytes), page_size=16, num_bytes_per_token=...)","handlingStrategy":"type-guard","validationCode":"u8 = src.tensor.view(torch.uint8)\nassert u8.dim() == 2, f\"source tensor must be 2-D as uint8, got {u8.dim()}-D\"","typeGuard":"def is_valid_kv_source(src) -> bool:\n    return src.tensor.is_contiguous() and src.tensor.view(torch.uint8).dim() == 2","tryCatchPattern":null,"preventionTips":["Always reshape cache tensors to [slots, slot_bytes] before wrapping in RealKvSource","Keep a single factory that normalizes source tensor shapes"],"tags":["kv-canary","shape-validation","tensor-layout"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}