{"record":{"id":"b315d0a10f54e30e","repo":"sgl-project/sglang","slug":"kv-canary-name-must-have-dtype-dtype-got-te","errorCode":null,"errorMessage":"kv-canary: {name} must have dtype {dtype}, got {tensor.dtype}","messagePattern":"kv-canary: (.+?) must have dtype (.+?), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/plan/utils.py","lineNumber":27,"sourceCode":"\ndef _resolve_swa_lut(\n    lut: Optional[torch.Tensor], device: torch.device\n) -> tuple[torch.Tensor, int, bool]:\n    \"\"\"Return the (tensor, length, has_lut) triple to launch the plan kernel with.\n\n    Triton requires a valid tensor pointer at every kernel-arg slot even when ``HAS_SWA_LUT`` is False, so\n    when the caller passes ``None`` we substitute a one-element sentinel tensor and set ``lut_len=0``;\n    the kernel's constexpr branch guarantees no dereference happens. Dtype matches the production LUT\n    (int64) so Triton ``tl.load`` element typing stays consistent.\n    \"\"\"\n    if lut is not None:\n        return lut, int(lut.shape[0]), True\n    return torch.zeros(1, dtype=torch.int64, device=device), 0, False\n\n\ndef _require_dtype(tensor: torch.Tensor, name: str, dtype: torch.dtype) -> None:\n    if tensor.dtype != dtype:\n        raise ValueError(\n            f\"kv-canary: {name} must have dtype {dtype}, got {tensor.dtype}\"\n        )\n\n\ndef _require_1d(tensor: torch.Tensor, name: str) -> None:\n    if tensor.ndim != 1:\n        raise ValueError(\n            f\"kv-canary: {name} must be 1-D, got shape {tuple(tensor.shape)}\"\n        )\n\n\ndef _require_2d(tensor: torch.Tensor, name: str) -> None:\n    if tensor.ndim != 2:\n        raise ValueError(\n            f\"kv-canary: {name} must be 2-D, got shape {tuple(tensor.shape)}\"\n        )\n\n","sourceCodeStart":9,"sourceCodeEnd":45,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/plan/utils.py#L9-L45","documentation":"Plan-kernel input tensors must have exactly the expected dtype (typically int32/int64) because the JIT kernels read raw typed memory without conversion. Any other dtype causes hard-to-debug garbage rather than an async CUDA fault.","triggerScenarios":"Calling the plan launch path with an input tensor whose dtype differs from the required one — e.g. int32 lens where int64 is required, or vice versa.","commonSituations":"Upstream code refactored lens/indices from int64 to int32 (or the reverse) for memory savings; mixing tensors created by different components with different dtype conventions.","solutions":["Cast the offending tensor to the required dtype before the call: t = t.to(torch.int64) (or int32 per the error text)","Standardize dtype conventions where the tensors are produced, not at the launch site","Check the error message for the tensor name to find which input is wrong"],"exampleFix":"// before\nprefix_lens = prefix_lens.to(torch.int32)  # kernel wants int64\nlaunch_plan_offsets_kernel(..., prefix_lens=prefix_lens, ...)\n// after\nprefix_lens = prefix_lens.to(torch.int64)\nlaunch_plan_offsets_kernel(..., prefix_lens=prefix_lens, ...)","handlingStrategy":"validation","validationCode":"expected = {'req_pool_indices': torch.int64, 'prefix_lens': torch.int64, 'extend_seq_lens': torch.int32}  # match kernel spec\nfor name, t, dt in inputs:\n    assert t.dtype == dt, f\"{name}: {t.dtype} != {dt}\"","typeGuard":"def has_dtype(t: torch.Tensor, dt: torch.dtype) -> bool:\n    return t.dtype == dt","tryCatchPattern":null,"preventionTips":["Cast inputs to the documented dtype at the boundary where they enter the plan path","Document dtype conventions for every tensor in the plan input dataclass"],"tags":["kv-canary","dtype","validation"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}