{"record":{"id":"f0981059ee236d49","repo":"sgl-project/sglang","slug":"qkv-weight-has-incompatible-output-dim-for-grouped","errorCode":null,"errorMessage":"qkv weight has incompatible output dim for grouped checkpoint layout: got {tuple(weight.shape)}, expected first dim {expected_out}.","messagePattern":"qkv weight has incompatible output dim for grouped checkpoint layout: got (.+?), expected first dim (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py","lineNumber":208,"sourceCode":"        \"img_pos_for_infer_output_info\",\n        \"local_embedding_layout\",\n        \"packed_seq_params\",\n        \"refiner_packed_seq_params\",\n    }\n)\n\n\ndef _reorder_grouped_qkv_to_qkv(\n    weight: torch.Tensor,\n    *,\n    num_query_groups: int,\n    heads_per_group: int,\n    head_dim: int,\n) -> torch.Tensor:\n    per_group = (heads_per_group + 2) * head_dim\n    expected_out = num_query_groups * per_group\n    if weight.shape[0] != expected_out:\n        raise ValueError(\n            \"qkv weight has incompatible output dim for grouped checkpoint layout: \"\n            f\"got {tuple(weight.shape)}, expected first dim {expected_out}.\"\n        )\n\n    rest_shape = weight.shape[1:]\n    grouped = weight.reshape(num_query_groups, per_group, *rest_shape)\n    q, k, v = torch.split(\n        grouped,\n        [heads_per_group * head_dim, head_dim, head_dim],\n        dim=1,\n    )\n    return torch.cat(\n        [\n            q.reshape(num_query_groups * heads_per_group * head_dim, *rest_shape),\n            k.reshape(num_query_groups * head_dim, *rest_shape),\n            v.reshape(num_query_groups * head_dim, *rest_shape),\n        ],\n        dim=0,","sourceCodeStart":190,"sourceCodeEnd":226,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py#L190-L226","documentation":"_reorder_grouped_qkv_to_qkv rewrites a grouped-QKV fused weight (per group: (heads_per_group + 2) * head_dim rows for q, k, v) into plain QKV order. It validates weight.shape[0] == num_query_groups * per_group; otherwise the checkpoint layout doesn't match the claimed grouping and reorder is aborted.","triggerScenarios":"Passing a fused qkv weight whose row count doesn't equal num_query_groups * (heads_per_group + 2) * head_dim — e.g. num_query_heads/num_kv_heads or head_dim misconfigured relative to the checkpoint, or a checkpoint already in plain QKV layout fed into the grouped-reorder path.","commonSituations":"GQA configs where heads_per_group is computed wrong (num_query_heads / num_kv_heads mismatch); loading a checkpoint with a different head layout than the config declares; test fixtures with synthetic shapes that don't match the grouping args.","solutions":["Verify the config's num_query_heads, num_kv_heads (hence heads_per_group) and head_dim against the checkpoint's qkv weight shape[0]","If the checkpoint is already in plain QKV (out = (num_q + 2*num_kv) * head_dim) skip the grouped reorder path","Fix the caller (_reorder_checkpoint_weight) to compute num_query_groups/per_group from the actual tensor shape"],"exampleFix":"# before\nreorder(w, num_query_groups=8, heads_per_group=1, head_dim=128)\n# w.shape[0] == (num_q + 2*num_kv) * 128 -> raises\n\n# after\nif w.shape[0] == (num_q + 2 * num_kv) * head_dim:\n    reordered = w  # already plain QKV layout\nelse:\n    reordered = reorder(w, num_query_groups, heads_per_group, head_dim)","handlingStrategy":"validation","validationCode":"per_group = (heads_per_group + 2) * head_dim\nexpected = num_query_groups * per_group\nis_grouped = weight.shape[0] == expected\nis_plain = weight.shape[0] == (num_query_groups*heads_per_group + 2*num_query_groups) * head_dim\nassert is_grouped or is_plain","typeGuard":"def is_grouped_qkv(w, ngrp, hpg, hd) -> bool:\n    return w.dim() >= 1 and w.shape[0] == ngrp * (hpg + 2) * hd","tryCatchPattern":null,"preventionTips":["Derive grouping args from checkpoint tensor shape, not config alone","Unit-test reorder with synthetic shapes matching real configs"],"tags":["qkv","gqa","checkpoint-loading","weight-reorder","shape-mismatch"],"backgroundTag":"weight-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}