{"record":{"id":"2a4291e2494adbb8","repo":"sgl-project/sglang","slug":"mimov2-fused-qkv-proj-checkpoint-is-tp-expected-f","errorCode":null,"errorMessage":"MiMoV2 fused qkv_proj checkpoint is TP={expected_fused_tp_size}-interleaved; got attention tp_size={tp_size} while loading {name}.","messagePattern":"MiMoV2 fused qkv_proj checkpoint is TP=(.+?)-interleaved; got attention tp_size=(.+?) while loading (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/models/mimo_v2.py","lineNumber":111,"sourceCode":"\n\ndef load_mimo_v2_qkv_proj_weight(\n    name,\n    param,\n    loaded_weight,\n    expected_fused_tp_size: Optional[int] = None,\n    deferred_scale_inv: Optional[Dict[str, torch.Tensor]] = None,\n):\n    tp_size = get_parallel().attn_tp_size\n    tp_rank = get_parallel().attn_tp_rank\n    ckpt_tp = expected_fused_tp_size if expected_fused_tp_size is not None else tp_size\n\n    if ckpt_tp == tp_size and loaded_weight.shape == param.shape:\n        default_weight_loader(param, loaded_weight)\n        return\n\n    if expected_fused_tp_size is not None and expected_fused_tp_size % tp_size != 0:\n        raise ValueError(\n            f\"MiMoV2 fused qkv_proj checkpoint is TP={expected_fused_tp_size}-\"\n            f\"interleaved; got attention tp_size={tp_size} while loading {name}.\"\n        )\n\n    is_scale_inv = \"weight_scale_inv\" in name\n\n    if is_scale_inv and ckpt_tp != tp_size:\n        if deferred_scale_inv is not None:\n            deferred_scale_inv[name] = loaded_weight.clone()\n            return\n        raise ValueError(\n            f\"qkv_proj scale_inv {name}: shape mismatch \"\n            f\"{tuple(loaded_weight.shape)} vs {tuple(param.shape)} \"\n            f\"due to block quantization ceiling; pass deferred_scale_inv dict\"\n        )\n\n    if loaded_weight.ndim != param.ndim or loaded_weight.shape[1:] != param.shape[1:]:\n        raise ValueError(","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/mimo_v2.py#L93-L129","documentation":"MiMoV2 checkpoints with fused qkv_proj weights are interleaved for a specific tensor-parallel size (expected_fused_tp_size). When loading, if the attention TP size doesn't evenly divide that checkpoint TP size, the weights can't be repartitioned and loading aborts. This guards against silently corrupt TP splits.","triggerScenarios":"Launching MiMoV2 with --tp-size (or attention TP shard) that does not divide the fused qkv checkpoint's interleave size, e.g. ckpt interleaved for TP=8 served with --tp 3. Raised from load_weights via load_mimo_v2_qkv_proj_weight when shapes don't match directly.","commonSituations":"Changing --tp-size on hardware that doesn't match how the fused checkpoint was exported; using attention-TP-subset parallelism on a fused-interleaved checkpoint; checkpoint exported with fused qkv at a TP size incompatible with the deployment.","solutions":["Relaunch with a TP size that evenly divides expected_fused_tp_size (e.g. 1, 2, 4, 8 for an 8-way interleaved ckpt)","If you control export, re-export the checkpoint without fused/interleaved qkv_proj or at your serving TP size","Check attention TP decomposition settings (attn_tp_size) so they align with the checkpoint layout"],"exampleFix":"# before\npython -m sglang.launch_server --model mimo-v2 --tp 3\n# after\npython -m sglang.launch_server --model mimo-v2 --tp 4","handlingStrategy":"validation","validationCode":"expected_fused_tp_size = 8  # from checkpoint metadata / MiMoV2 export\ntp_size = server_args.tp_size\nassert expected_fused_tp_size is None or expected_fused_tp_size % tp_size == 0, (\n    f\"tp={tp_size} incompatible with fused qkv ckpt TP={expected_fused_tp_size}\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Restrict --tp-size to powers of two dividing the checkpoint's export TP","Re-export checkpoints at your deployment TP size if you control conversion","Document the fused interleave size alongside the checkpoint"],"tags":["mimo-v2","tensor-parallel","weight-loading","checkpoint-layout"],"backgroundTag":"tensor-parallel-size-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}