{"record":{"id":"8fb29724af113cd2","repo":"sgl-project/sglang","slug":"qkv-proj-weight-name-unexpected-shape-tuple-lo-8fb297","errorCode":null,"errorMessage":"qkv_proj weight {name}: unexpected shape {tuple(loaded_weight.shape)}; expected fused {fused_shape} or sharded {tuple(param.shape)}","messagePattern":"qkv_proj weight (.+?): unexpected shape (.+?); expected fused (.+?) or sharded (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/mimo_v2.py","lineNumber":137,"sourceCode":"        if deferred_scale_inv is not None:\n            deferred_scale_inv[name] = loaded_weight.clone()\n            return\n        raise ValueError(\n            f\"qkv_proj scale_inv {name}: shape mismatch \"\n            f\"{tuple(loaded_weight.shape)} vs {tuple(param.shape)} \"\n            f\"due to block quantization ceiling; pass deferred_scale_inv dict\"\n        )\n\n    if loaded_weight.ndim != param.ndim or loaded_weight.shape[1:] != param.shape[1:]:\n        raise ValueError(\n            f\"qkv_proj weight {name}: unexpected shape {tuple(loaded_weight.shape)}; \"\n            f\"expected sharded {tuple(param.shape)}\"\n        )\n\n    if tp_size == ckpt_tp:\n        fused_shape = (param.shape[0] * tp_size, *param.shape[1:])\n        if tuple(loaded_weight.shape) != fused_shape:\n            raise ValueError(\n                f\"qkv_proj weight {name}: unexpected shape \"\n                f\"{tuple(loaded_weight.shape)}; expected fused {fused_shape} \"\n                f\"or sharded {tuple(param.shape)}\"\n            )\n        default_weight_loader(param, loaded_weight.chunk(tp_size, dim=0)[tp_rank])\n    else:\n        shards_per_rank = ckpt_tp // tp_size\n        shards = loaded_weight.chunk(ckpt_tp, dim=0)\n        merged = torch.cat(\n            shards[tp_rank * shards_per_rank : (tp_rank + 1) * shards_per_rank],\n            dim=0,\n        )\n        default_weight_loader(param, merged)\n\n\ndef _get_ckpt_qkv_shard_sizes(config, layer_name, ckpt_tp):\n    m = re.search(r\"layers\\.(\\d+)\\.\", layer_name)\n    if m is None:","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/mimo_v2.py#L119-L155","documentation":"When runtime tp_size equals the checkpoint tp, the loaded fused qkv weight must be exactly param.shape[0]*tp_size rows. This error means the fused tensor row count does not match tp_size * per-rank shard, so it can neither be chunked nor used as-is.","triggerScenarios":"tp_size == ckpt_tp but tuple(loaded_weight.shape) != (param.shape[0]*tp_size, *param.shape[1:]) — e.g. num_kv_heads handling differs between checkpoint and config, producing a wrong fused row count (hidden + 2*kv rows).","commonSituations":"Mismatched num_key_value_heads between checkpoint and config, partially converted checkpoints, or GQA checkpoints where kv head count doesn't divide tp_size the same way.","solutions":["Check num_attention_heads and num_key_value_heads in config match the checkpoint's fused qkv shape (out = hidden + 2*kv_dim)","Re-run with a tp_size that divides the number of KV heads, or fix the config","Re-export the checkpoint with correct fused qkv ordering (interleaved q/k/v)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"c = config\nkv = c.num_key_value_heads\nexpected_rows = c.hidden_size + 2 * kv * c.head_dim\nif tuple(w.shape)[0] != expected_rows * tp_size // tp_size and tp == ckpt_tp:\n    raise SystemExit('qkv fused shape mismatch; check num_kv_heads')","typeGuard":"def fused_qkv_ok(w: torch.Tensor, param: torch.Tensor, tp: int) -> bool:\n    return tuple(w.shape) == (param.shape[0] * tp, *param.shape[1:])","tryCatchPattern":null,"preventionTips":["Ensure num_key_value_heads is divisible by tp_size","Pre-validate checkpoint qkv tensor shapes against model config"],"tags":["weight-loading","shape-mismatch","gqa","tensor-parallel"],"backgroundTag":"weight-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}