{"record":{"id":"bfbf054f5266004b","repo":"sgl-project/sglang","slug":"invalid-gate-up-proj-shape-for-name-tuple-load","errorCode":null,"errorMessage":"Invalid gate_up_proj shape for {name}: {tuple(loaded_weight.shape)}","messagePattern":"Invalid gate_up_proj shape for (.+?): (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/models/lfm2_moe.py","lineNumber":624,"sourceCode":"            # FusedMoE layout, so no transpose is needed.\n            if \"feed_forward.experts.gate_up_proj\" in name:\n                fused_name = name\n                if fused_name.endswith(\".weight\"):\n                    fused_name = fused_name[: -len(\".weight\")]\n                fused_name = fused_name.replace(\n                    \"feed_forward.experts.gate_up_proj\",\n                    \"feed_forward.experts.w13_weight\",\n                )\n                if fused_name in params_dict:\n                    if loaded_weight.dim() != 3:\n                        raise ValueError(\n                            f\"Expected a 3D packed tensor for {name}, got \"\n                            f\"{loaded_weight.dim()}D {tuple(loaded_weight.shape)}\"\n                        )\n                    param = params_dict[fused_name]\n                    weight_loader = param.weight_loader\n                    if loaded_weight.shape[1] % 2 != 0:\n                        raise ValueError(\n                            f\"Invalid gate_up_proj shape for {name}: \"\n                            f\"{tuple(loaded_weight.shape)}\"\n                        )\n                    w1, w3 = loaded_weight.chunk(2, dim=1)\n                    for expert_id in range(w1.shape[0]):\n                        weight_loader(\n                            param,\n                            w1[expert_id],\n                            fused_name,\n                            shard_id=\"w1\",\n                            expert_id=expert_id,\n                        )\n                        weight_loader(\n                            param,\n                            w3[expert_id],\n                            fused_name,\n                            shard_id=\"w3\",\n                            expert_id=expert_id,","sourceCodeStart":606,"sourceCodeEnd":642,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/lfm2_moe.py#L606-L642","documentation":"Before chunking the packed gate_up tensor into w1/w3 along dim 1, LFM2-MoE checks loaded_weight.shape[1] % 2 == 0 (lfm2_moe.py:624). An odd second dimension cannot be evenly split into gate and up projections, so the tensor is malformed for the fused layout and loading aborts.","triggerScenarios":"A gate_up_proj checkpoint tensor with odd intermediate dimension (e.g. shape [E, 1101, H]) reaching the w13 loader - usually a botched conversion or truncated tensor.","commonSituations":"Manual safetensors surgery, quantization tooling that rounds intermediate sizes, corrupted downloads.","solutions":["Re-download / regenerate the checkpoint with correct fused shapes ([E, 2*I, H])","Validate shape[1] is even and equals 2*config.intermediate_size before loading","Report to the checkpoint publisher if the shipped file is malformed"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert w.dim() == 3 and w.shape[1] == 2 * cfg.intermediate_size and w.shape[1] % 2 == 0","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate fused-tensor shapes against config after any conversion/quantization step"],"tags":["lfm2","moe","weight-loading","shape-validation"],"backgroundTag":"checkpoint-tensor-rank-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}