sgl-project/sglang · error · ValueError

qkv weight has incompatible output dim for grouped checkpoin

Error message

qkv weight has incompatible output dim for grouped checkpoint layout: got {tuple(weight.shape)}, expected first dim {expected_out}.

What it means

_reorder_grouped_qkv_to_qkv rewrites a grouped-QKV fused weight (per group: (heads_per_group + 2) * head_dim rows for q, k, v) into plain QKV order. It validates weight.shape[0] == num_query_groups * per_group; otherwise the checkpoint layout doesn't match the claimed grouping and reorder is aborted.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py:208

        "img_pos_for_infer_output_info",
        "local_embedding_layout",
        "packed_seq_params",
        "refiner_packed_seq_params",
    }
)


def _reorder_grouped_qkv_to_qkv(
    weight: torch.Tensor,
    *,
    num_query_groups: int,
    heads_per_group: int,
    head_dim: int,
) -> torch.Tensor:
    per_group = (heads_per_group + 2) * head_dim
    expected_out = num_query_groups * per_group
    if weight.shape[0] != expected_out:
        raise ValueError(
            "qkv weight has incompatible output dim for grouped checkpoint layout: "
            f"got {tuple(weight.shape)}, expected first dim {expected_out}."
        )

    rest_shape = weight.shape[1:]
    grouped = weight.reshape(num_query_groups, per_group, *rest_shape)
    q, k, v = torch.split(
        grouped,
        [heads_per_group * head_dim, head_dim, head_dim],
        dim=1,
    )
    return torch.cat(
        [
            q.reshape(num_query_groups * heads_per_group * head_dim, *rest_shape),
            k.reshape(num_query_groups * head_dim, *rest_shape),
            v.reshape(num_query_groups * head_dim, *rest_shape),
        ],
        dim=0,

View on GitHub (pinned to 0132848349)

Solutions

1. Verify the config's num_query_heads, num_kv_heads (hence heads_per_group) and head_dim against the checkpoint's qkv weight shape[0]
2. If the checkpoint is already in plain QKV (out = (num_q + 2*num_kv) * head_dim) skip the grouped reorder path
3. Fix the caller (_reorder_checkpoint_weight) to compute num_query_groups/per_group from the actual tensor shape

Example fix

# before
reorder(w, num_query_groups=8, heads_per_group=1, head_dim=128)
# w.shape[0] == (num_q + 2*num_kv) * 128 -> raises

# after
if w.shape[0] == (num_q + 2 * num_kv) * head_dim:
    reordered = w  # already plain QKV layout
else:
    reordered = reorder(w, num_query_groups, heads_per_group, head_dim)

per_group = (heads_per_group + 2) * head_dim
expected = num_query_groups * per_group
is_grouped = weight.shape[0] == expected
is_plain = weight.shape[0] == (num_query_groups*heads_per_group + 2*num_query_groups) * head_dim
assert is_grouped or is_plain

def is_grouped_qkv(w, ngrp, hpg, hd) -> bool:
    return w.dim() >= 1 and w.shape[0] == ngrp * (hpg + 2) * hd

When it happens

Trigger: Passing a fused qkv weight whose row count doesn't equal num_query_groups * (heads_per_group + 2) * head_dim — e.g. num_query_heads/num_kv_heads or head_dim misconfigured relative to the checkpoint, or a checkpoint already in plain QKV layout fed into the grouped-reorder path.

Common situations: GQA configs where heads_per_group is computed wrong (num_query_heads / num_kv_heads mismatch); loading a checkpoint with a different head layout than the config declares; test fixtures with synthetic shapes that don't match the grouping args.

Related errors

• {debug_name} requires cache_head_start when cache heads ({nu
• qkv_proj weight {name}: unexpected shape {tuple(loaded_weigh
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q
• k shape must be [num_tokens, num_kv_heads, head_dim], got {k
• k_pool has incompatible shape {k_pool.shape}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f0981059ee236d49. Report an issue: GitHub.