{"record":{"id":"cf062935f43e2c5b","repo":"sgl-project/sglang","slug":"invalid-packed-q-size-q-dim-must-be-divisible-b-cf0629","errorCode":null,"errorMessage":"Invalid packed Q size {q_dim}: must be divisible by K={K}. KDA packed decode requires num_q_heads == num_k_heads and head_q_dim == head_k_dim.","messagePattern":"Invalid packed Q size (.+?): must be divisible by K=(.+?)\\. KDA packed decode requires num_q_heads == num_k_heads and head_q_dim == head_k_dim\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent.py","lineNumber":615,"sourceCode":"        raise ValueError(f\"`A_log` must have {HV} elements (got {A_log.numel()}).\")\n    if dt_bias.numel() != HV * K:\n        raise ValueError(\n            f\"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()}).\"\n        )\n    if out.shape != (B, 1, HV, V):\n        raise ValueError(\n            f\"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)}).\"\n        )\n\n    qkv_dim = mixed_qkv.shape[1]\n    qk_dim = qkv_dim - HV * V\n    if qk_dim <= 0 or qk_dim % 2 != 0:\n        raise ValueError(\n            f\"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}.\"\n        )\n    q_dim = qk_dim // 2\n    if q_dim % K != 0:\n        raise ValueError(\n            f\"Invalid packed Q size {q_dim}: must be divisible by K={K}. \"\n            \"KDA packed decode requires num_q_heads == num_k_heads and \"\n            \"head_q_dim == head_k_dim.\"\n        )\n    H = q_dim // K\n    if H <= 0 or HV % H != 0:\n        raise ValueError(\n            f\"Invalid head config inferred from mixed_qkv: H={H}, HV={HV}.\"\n        )\n\n    # Batched-decode CUDA fast path:\n    # row-streaming state update reaches the in-place R+W bandwidth of the\n    # part (~9.6 TB/s) where this triton kernel tops out at ~5 TB/s holding a\n    # [BV, K] register tile per warp. ULP-level output differences only\n    # (reduction order); small batches keep triton (launch-bound anyway).\n    if use_qk_l2norm_in_kernel:\n        from sglang.kernels.ops.attention import kda_packed_decode as kda_decode_cuda\n","sourceCodeStart":597,"sourceCodeEnd":633,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent.py#L597-L633","documentation":"After splitting the QK half of packed mixed_qkv, the Q size (qk_dim//2) must be divisible by K (head_k_dim). KDA packed decode only supports num_q_heads == num_k_heads and head_q_dim == head_k_dim, so q_dim must be an exact multiple of K.","triggerScenarios":"Calling packed decode with GQA-style KDA weights where num_q_heads is a multiple-but-not-equal of num_k_heads, or head_q_dim != head_k_dim, making q_dim % K != 0.","commonSituations":"Reusing a chunked prefill kernel config on the packed decode path; loading a KDA checkpoint with q/k head dim mismatch (e.g. 128 vs 64); hand-crafting mixed_qkv for tests.","solutions":["Use the non-packed decode path (fused_recurrent_kda with unpacked q/k/v) for GQA-shaped models","Verify head_q_dim == head_k_dim and num_q_heads == num_k_heads in the checkpoint config","Repack or project q to have exactly H*K elements matching k"],"exampleFix":"// before\nout = fused_recurrent_kda_packed_decode(mixed_qkv, ...)  # GQA weights, q_dim % K != 0\n// after\nq, k, v = unpack(mixed_qkv)  # use unpacked API for GQA geometry\nout = fused_recurrent_kda(q=q, k=k, v=v, ...)","handlingStrategy":"type-guard","validationCode":"assert num_q_heads == num_k_heads and head_q_dim == head_k_dim, 'packed decode needs MHA-style q/k'","typeGuard":"def supports_packed_decode(cfg) -> bool:\n    return cfg.num_q_heads == cfg.num_k_heads and cfg.head_q_dim == cfg.head_k_dim","tryCatchPattern":"try:\n    out = fused_recurrent_kda_packed_decode(...)\nexcept ValueError:\n    out = fused_recurrent_kda(q, k, v, ...)  # unpacked fallback","preventionTips":["Branch on head geometry at model load: packed fast path only for MHA-shaped KDA","Keep a GQA fallback path wired in"],"tags":["pytorch","tensor-shape","gqa","kda","packed-qkv"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}