{"record":{"id":"f9ef3ea32fd702a3","repo":"sgl-project/sglang","slug":"gate-name-has-shape-tuple-g-shape-expected","errorCode":null,"errorMessage":"{gate_name} has shape {tuple(g.shape)}, expected ({H * D}, {hidden})","messagePattern":"(.+?) has shape (.+?), expected \\((.+?), (.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/hardware_backend/mlx/models/muse_glimmer_mlx.py","lineNumber":703,"sourceCode":"            if \"rotary_emb\" in name:\n                continue\n            if any(marker in name for marker in _VISION_KEY_MARKERS):\n                continue\n            # Consumed below when its q_proj comes up.\n            if name.endswith(\"output_gate_proj.weight\"):\n                continue\n\n            # The reference computes rms_norm(x, weight + 1.0) for these four\n            # norms; fold the +1 so plain nn.RMSNorm is exact.  model.norm\n            # (MuseGlimmerFinalRMSNorm) is deliberately not offset.\n            if name.endswith(_OFFSET_NORM_SUFFIXES):\n                w = w + 1.0\n\n            if name.endswith(\"q_proj.weight\") and self.args.use_attn_output_gate:\n                gate_name = name.replace(\"q_proj.weight\", \"output_gate_proj.weight\")\n                g = weights[gate_name]\n                if tuple(g.shape) != (H * D, hidden):\n                    raise ValueError(\n                        f\"{gate_name} has shape {tuple(g.shape)}, expected \"\n                        f\"({H * D}, {hidden})\"\n                    )\n                # Per-head interleave [q_head; gate_head]: (H*D, hidden) x2\n                # -> (H, 2D, hidden) -> (2*H*D, hidden).\n                w = mx.concatenate(\n                    [w.reshape(H, D, hidden), g.reshape(H, D, hidden)], axis=1\n                ).reshape(2 * H * D, hidden)\n\n            new_weights[name] = w\n\n        return new_weights\n\n\ndef _normalize_rc_layout(weights: dict) -> dict:\n    \"\"\"Rewrite RC multimodal keys to the raw text-only schema.\n\n    Drops the vision tower/adapter/projection, strips the","sourceCodeStart":685,"sourceCodeEnd":721,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/hardware_backend/mlx/models/muse_glimmer_mlx.py#L685-L721","documentation":"When use_attn_output_gate is enabled, each layer's output_gate_proj.weight must have shape (num_attention_heads * head_dim, hidden_size) before being interleaved with q_proj per head. A gate tensor of any other shape fails this per-key check during sanitize.","triggerScenarios":"Loading a raw checkpoint where an output_gate_proj.weight tensor has an unexpected shape — e.g. head_dim or num_attention_heads changed in config after export, or a corrupted/mis-shaped gate shard.","commonSituations":"Editing head_dim/num_attention_heads in config.json without matching weights, mixing per-tensor-quantized artifacts that altered dims, or a bad shard merge producing wrong gate shapes.","solutions":["Restore config.json's num_attention_heads/head_dim to the values the checkpoint was exported with","Verify each layer's output_gate_proj.weight shape: should be (H*D, hidden)","Re-download the checkpoint if the tensor itself is corrupt"],"exampleFix":"# before: config head_dim=128 -> expects (32*128, 4096) but tensor is (2048, 4096)\n# after: set \"head_dim\": 64 (so H*D == 2048) to match the real tensors","handlingStrategy":"validation","validationCode":"H, D, hid = cfg[\"num_attention_heads\"], cfg[\"head_dim\"], cfg[\"hidden_size\"]\nfor k, g in weights.items():\n    if k.endswith(\"output_gate_proj.weight\"):\n        assert tuple(g.shape) == (H * D, hid), k","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't edit head_dim/num_attention_heads post-export","Spot-check per-layer tensor shapes before full load"],"tags":["shape-mismatch","attention-gate","weight-loading"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}