{"record":{"id":"fb3b8fd6775bec86","repo":"sgl-project/sglang","slug":"f-sglang-kda-state-must-be-b-h-v-k-with-v-k-e","errorCode":null,"errorMessage":"f\"SGLang KDA state must be [B,H,V,K] with (V,K)={expected}, got {tuple(state.shape)}\"","messagePattern":"f\"SGLang KDA state must be \\[B,H,V,K\\] with \\(V,K\\)=(.+?), got (.+?)\"","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/linear/kernels/kda_nvidia.py","lineNumber":54,"sourceCode":"\nfrom sglang.srt.layers.attention.linear.kernels.kda_triton import TritonKDAKernel\nfrom sglang.srt.layers.attention.linear.kernels.kernel_backend import (\n    LinearAttnKernelBase,\n)\n\nlogger = logging.getLogger(__name__)\n\n_BUCKETS = (2048, 4096, 8192, 16384)\n_MAX_NVIDIA_KDA_BATCH = 8\n\n\ndef _to_nvidia_kda_state_layout(\n    state: torch.Tensor, *, head_k_dim: int, head_v_dim: int\n) -> torch.Tensor:\n    \"\"\"Materialize SGLang [B,H,V,K] state as vendor [B,H,K,V].\"\"\"\n    expected = (head_v_dim, head_k_dim)\n    if state.ndim != 4 or tuple(state.shape[-2:]) != expected:\n        raise ValueError(\n            \"SGLang KDA state must be [B,H,V,K] with \"\n            f\"(V,K)={expected}, got {tuple(state.shape)}\"\n        )\n    return state.transpose(-1, -2).float().contiguous()\n\n\ndef _from_nvidia_kda_state_layout(\n    state: torch.Tensor,\n    *,\n    head_k_dim: int,\n    head_v_dim: int,\n    dtype: torch.dtype,\n) -> torch.Tensor:\n    \"\"\"Materialize vendor [B,H,K,V] state as SGLang [B,H,V,K].\"\"\"\n    expected = (head_k_dim, head_v_dim)\n    if state.ndim != 4 or tuple(state.shape[-2:]) != expected:\n        raise ValueError(\n            \"NVIDIA KDA state must be [B,H,K,V] with \"","sourceCodeStart":36,"sourceCodeEnd":72,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/linear/kernels/kda_nvidia.py#L36-L72","documentation":"_to_nvidia_kda_state_layout validates that SGLang-format state is 4-D [B,H,V,K] with the last two dims exactly (head_v_dim, head_k_dim) before transposing to the vendor [B,H,K,V] layout. Any other shape raises ValueError.","triggerScenarios":"Calling extend on NvidiaKDAKernel (or the layout unit tests) with a state tensor whose trailing dims are swapped (K,V) or whose ndim != 4.","commonSituations":"Passing a vendor-layout state straight back in, or a cache allocated with head dims in the wrong order; refactors of the state pool layout.","solutions":["Ensure the input state is [B,H,V,K] with V=head_v_dim, K=head_k_dim","If you hold a [B,H,K,V] tensor, convert with the inverse helper first","Assert the trailing shape before calling extend"],"exampleFix":"# before\nstate = torch.empty(B, H, K, V)  # vendor layout passed directly\n# after\nstate = torch.empty(B, H, V, K)  # SGLang layout","handlingStrategy":"validation","validationCode":"assert state.ndim == 4 and tuple(state.shape[-2:]) == (head_v_dim, head_k_dim), (\n    f'expected [B,H,V,K] ending ({head_v_dim},{head_k_dim}), got {tuple(state.shape)}')","typeGuard":"def is_sglang_kda_state(t: torch.Tensor, head_v_dim: int, head_k_dim: int) -> bool:\n    return t.ndim == 4 and tuple(t.shape[-2:]) == (head_v_dim, head_k_dim)","tryCatchPattern":null,"preventionTips":["Tag tensors with their layout convention in custom code","Use the provided to/from layout helpers instead of manual transposes"],"tags":["sglang","nvidia","kda","tensor-shape","layout"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}