sgl-project/sglang · error · ValueError

raw q_proj.weight has shape {raw_q_shape}, expected ({H * D}

Error message

raw q_proj.weight has shape {raw_q_shape}, expected ({H * D}, {hidden}); a width of {2 * H * D} means the gate is already fused — such artifacts must carry "muse_glimmer_mlx_format": {MUSE_GLIMMER_MLX_FORMAT_VERSION} in config.json

What it means

For a raw (non-packaged) checkpoint, q_proj.weight must have shape (num_attention_heads * head_dim, hidden_size). If the observed width is exactly 2*H*D, the output gate has already been fused into q_proj, which means the artifact is packaged and its config.json must declare muse_glimmer_mlx_format.

Source

Thrown at python/sglang/srt/hardware_backend/mlx/models/muse_glimmer_mlx.py:675

                f"{len(missing)} missing keys {missing[:4]}"
                f"{'...' if len(missing) > 4 else ''}, "
                f"{len(unexpected)} unexpected keys {unexpected[:4]}"
                f"{'...' if len(unexpected) > 4 else ''}{hint}"
            )

        H = self.args.num_attention_heads
        D = self.args.head_dim
        hidden = self.args.hidden_size

        embed_shape = tuple(weights["model.embed_tokens.weight"].shape)
        if embed_shape != (self.args.vocab_size, hidden):
            raise ValueError(
                f"embed_tokens.weight has shape {embed_shape} but config says "
                f"(vocab_size, hidden_size) = ({self.args.vocab_size}, {hidden})"
            )
        raw_q_shape = tuple(weights["model.layers.0.self_attn.q_proj.weight"].shape)
        if raw_q_shape != (H * D, hidden):
            raise ValueError(
                f"raw q_proj.weight has shape {raw_q_shape}, expected "
                f"({H * D}, {hidden}); a width of {2 * H * D} means the gate "
                "is already fused — such artifacts must carry "
                f'"muse_glimmer_mlx_format": {MUSE_GLIMMER_MLX_FORMAT_VERSION} in config.json'
            )

        new_weights = {}
        for name, w in weights.items():
            # mlx derives RoPE itself; drop cached buffers.
            if "rotary_emb" in name:
                continue
            if any(marker in name for marker in _VISION_KEY_MARKERS):
                continue
            # Consumed below when its q_proj comes up.
            if name.endswith("output_gate_proj.weight"):
                continue

            # The reference computes rms_norm(x, weight + 1.0) for these four

View on GitHub (pinned to 0132848349)

Solutions

1. Add \"muse_glimmer_mlx_format\": <MUSE_GLIMMER_MLX_FORMAT_VERSION> to the artifact's config.json
2. Or use the original raw HF export where q_proj is unfused
3. Or repackage correctly so config and weights agree

Example fix

// before: fused q_proj (8192, 4096) with no marker in config.json
// after: config.json gains
"muse_glimmer_mlx_format": 1  // = MUSE_GLIMMER_MLX_FORMAT_VERSION

H, D, hid = cfg["num_attention_heads"], cfg["head_dim"], cfg["hidden_size"]
q = weights["model.layers.0.self_attn.q_proj.weight"].shape
if q[0] == 2 * H * D:
    assert cfg.get("muse_glimmer_mlx_format") is not None, "fused weights need format marker"

try:
    w = sanitize(weights)
except ValueError as e:
    if "already fused" in str(e):
        cfg["muse_glimmer_mlx_format"] = VERSION; w = sanitize(weights)

When it happens

Trigger: Loading fused q_proj weights (width 2*H*D) without the muse_glimmer_mlx_format marker in config.json, so sanitize takes the raw path and rejects the fused tensor shape.

Common situations: Downloading a pre-fused MLX artifact whose config.json is missing or stripped of the format marker, or renaming/repacking dirs and losing the marker.

Related errors

• {tensor_name}{context_clause} with shape {tensor.shape} cann
• Block sparse tensors{context} must have shapes (B, H, M) and
• Block sparse tensors{context} {dim_name} dim must be {tgt} o
• Block sparse tensors{context} must share the same m-block di
• Block sparse tensors{context} n-block dimension must be <= {

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8c99a668999506ff. Report an issue: GitHub.