sgl-project/sglang · critical · ValueError

MiMoV2 fused qkv_proj checkpoint is TP={expected_fused_tp_si

Error message

MiMoV2 fused qkv_proj checkpoint is TP={expected_fused_tp_size}-interleaved; got attention tp_size={tp_size} while loading {name}.

What it means

MiMoV2 checkpoints with fused qkv_proj weights are interleaved for a specific tensor-parallel size (expected_fused_tp_size). When loading, if the attention TP size doesn't evenly divide that checkpoint TP size, the weights can't be repartitioned and loading aborts. This guards against silently corrupt TP splits.

Source

Thrown at python/sglang/srt/models/mimo_v2.py:111


def load_mimo_v2_qkv_proj_weight(
    name,
    param,
    loaded_weight,
    expected_fused_tp_size: Optional[int] = None,
    deferred_scale_inv: Optional[Dict[str, torch.Tensor]] = None,
):
    tp_size = get_parallel().attn_tp_size
    tp_rank = get_parallel().attn_tp_rank
    ckpt_tp = expected_fused_tp_size if expected_fused_tp_size is not None else tp_size

    if ckpt_tp == tp_size and loaded_weight.shape == param.shape:
        default_weight_loader(param, loaded_weight)
        return

    if expected_fused_tp_size is not None and expected_fused_tp_size % tp_size != 0:
        raise ValueError(
            f"MiMoV2 fused qkv_proj checkpoint is TP={expected_fused_tp_size}-"
            f"interleaved; got attention tp_size={tp_size} while loading {name}."
        )

    is_scale_inv = "weight_scale_inv" in name

    if is_scale_inv and ckpt_tp != tp_size:
        if deferred_scale_inv is not None:
            deferred_scale_inv[name] = loaded_weight.clone()
            return
        raise ValueError(
            f"qkv_proj scale_inv {name}: shape mismatch "
            f"{tuple(loaded_weight.shape)} vs {tuple(param.shape)} "
            f"due to block quantization ceiling; pass deferred_scale_inv dict"
        )

    if loaded_weight.ndim != param.ndim or loaded_weight.shape[1:] != param.shape[1:]:
        raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

Relaunch with a TP size that evenly divides expected_fused_tp_size (e.g. 1, 2, 4, 8 for an 8-way interleaved ckpt)
If you control export, re-export the checkpoint without fused/interleaved qkv_proj or at your serving TP size
Check attention TP decomposition settings (attn_tp_size) so they align with the checkpoint layout

Example fix

# before
python -m sglang.launch_server --model mimo-v2 --tp 3
# after
python -m sglang.launch_server --model mimo-v2 --tp 4

Defensive patterns

Strategy: validation

Validate before calling

expected_fused_tp_size = 8  # from checkpoint metadata / MiMoV2 export
tp_size = server_args.tp_size
assert expected_fused_tp_size is None or expected_fused_tp_size % tp_size == 0, (
    f"tp={tp_size} incompatible with fused qkv ckpt TP={expected_fused_tp_size}")

Prevention

Restrict --tp-size to powers of two dividing the checkpoint's export TP
Re-export checkpoints at your deployment TP size if you control conversion
Document the fused interleave size alongside the checkpoint

When it happens

Trigger: Launching MiMoV2 with --tp-size (or attention TP shard) that does not divide the fused qkv checkpoint's interleave size, e.g. ckpt interleaved for TP=8 served with --tp 3. Raised from load_weights via load_mimo_v2_qkv_proj_weight when shapes don't match directly.

Common situations: Changing --tp-size on hardware that doesn't match how the fused checkpoint was exported; using attention-TP-subset parallelism on a fused-interleaved checkpoint; checkpoint exported with fused qkv at a TP size incompatible with the deployment.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2a4291e2494adbb8. Report an issue: GitHub.