{"record":{"id":"9bbbca9f49773f9a","repo":"sgl-project/sglang","slug":"qkv-proj-scale-inv-name-shape-mismatch-tuple-l","errorCode":null,"errorMessage":"qkv_proj scale_inv {name}: shape mismatch {tuple(loaded_weight.shape)} vs {tuple(param.shape)} due to block quantization ceiling; pass deferred_scale_inv dict","messagePattern":"qkv_proj scale_inv (.+?): shape mismatch (.+?) vs (.+?) due to block quantization ceiling; pass deferred_scale_inv dict","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/mimo_v2.py","lineNumber":122,"sourceCode":"    ckpt_tp = expected_fused_tp_size if expected_fused_tp_size is not None else tp_size\n\n    if ckpt_tp == tp_size and loaded_weight.shape == param.shape:\n        default_weight_loader(param, loaded_weight)\n        return\n\n    if expected_fused_tp_size is not None and expected_fused_tp_size % tp_size != 0:\n        raise ValueError(\n            f\"MiMoV2 fused qkv_proj checkpoint is TP={expected_fused_tp_size}-\"\n            f\"interleaved; got attention tp_size={tp_size} while loading {name}.\"\n        )\n\n    is_scale_inv = \"weight_scale_inv\" in name\n\n    if is_scale_inv and ckpt_tp != tp_size:\n        if deferred_scale_inv is not None:\n            deferred_scale_inv[name] = loaded_weight.clone()\n            return\n        raise ValueError(\n            f\"qkv_proj scale_inv {name}: shape mismatch \"\n            f\"{tuple(loaded_weight.shape)} vs {tuple(param.shape)} \"\n            f\"due to block quantization ceiling; pass deferred_scale_inv dict\"\n        )\n\n    if loaded_weight.ndim != param.ndim or loaded_weight.shape[1:] != param.shape[1:]:\n        raise ValueError(\n            f\"qkv_proj weight {name}: unexpected shape {tuple(loaded_weight.shape)}; \"\n            f\"expected sharded {tuple(param.shape)}\"\n        )\n\n    if tp_size == ckpt_tp:\n        fused_shape = (param.shape[0] * tp_size, *param.shape[1:])\n        if tuple(loaded_weight.shape) != fused_shape:\n            raise ValueError(\n                f\"qkv_proj weight {name}: unexpected shape \"\n                f\"{tuple(loaded_weight.shape)}; expected fused {fused_shape} \"\n                f\"or sharded {tuple(param.shape)}\"","sourceCodeStart":104,"sourceCodeEnd":140,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/mimo_v2.py#L104-L140","documentation":"Raised while loading qkv_proj weight_scale_inv tensors for MiMo-v2 when the checkpoint's tensor-parallel degree differs from the runtime tp_size. Block-quantized scale_inv tensors cannot be naively resharded because their spatial blocks only cover a ceiling of rows, so the loader requires the caller to pass a deferred_scale_inv dict to collect them for later resolution.","triggerScenarios":"Loading a block-quantized (e.g. AWQ/GPTQ-style weight_scale_inv) MiMo-v2 checkpoint with a tp_size different from ckpt_tp in load_mimo_v2_qkv_proj_weight without passing deferred_scale_inv to load_weights.","commonSituations":"Running tp=2 or tp=4 on a checkpoint saved for tp=1 (or vice versa) with block quantization; custom weight-loading code that calls the loader directly without the dict.","solutions":["Pass a dict as deferred_scale_inv when calling load_mimo_v2_qkv_proj_weight / load_weights so scale_inv tensors are collected and resolved later by _resolve_deferred_qkv_scale_inv","Ensure you use the standard model.load_weights path (ModelRunner) which already threads deferred_scale_inv through","Match runtime tp_size to the checkpoint's fused tp size so is_scale_inv resharding is skipped"],"exampleFix":"# before\nmodel.load_weights(weights)  # scale_inv path raises\n\n# after\ndeferred = {}\nmodel.load_weights(weights, deferred_scale_inv=deferred)\n_resolve_deferred_qkv_scale_inv(model, deferred, ...)","handlingStrategy":"validation","validationCode":"expected = getattr(loaded, 'tp_size', 1)\nif 'weight_scale_inv' in name and expected != runtime_tp:\n    deferred = deferred or {}\n    deferred[name] = loaded\n    return","typeGuard":"def needs_deferred_scale_inv(name: str, ckpt_tp: int, tp: int) -> bool:\n    return name.endswith('weight_scale_inv') and ckpt_tp != tp","tryCatchPattern":null,"preventionTips":["Always allocate and pass deferred_scale_inv when loading block-quantized checkpoints under TP","Use the standard ModelRunner loading path rather than hand-rolled loops","Log ckpt_tp vs tp_size before loading to catch mismatch early"],"tags":["quantization","tensor-parallel","weight-loading","mimo"],"backgroundTag":"checkpoint-tp-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}