sgl-project/sglang · critical · ValueError

MiMoV2ForCausalLM requires effective attention TP size {expe

Error message

MiMoV2ForCausalLM requires effective attention TP size {expected_attn_tp_size} because its fused qkv_proj weights are TP={expected_attn_tp_size}-interleaved; got {effective_attn_tp_size} (tp_size={cfg.tp_size}, dp_size={cfg.dp_size}, enable_dp_attention={view.enable_dp_attention}, attn_cp_size={view.attn_cp_size}). 

What it means

MiMoV2ForCausalLM ships its fused qkv_proj weights pre-sharded/interleaved for a fixed attention tensor-parallel degree. SGLang computes the effective attention TP size as tp_size // attn_dp_size // attn_cp_size and rejects any value that does not evenly divide the weight's expected interleaving factor, since loading would produce corrupt attention projections.

Source

Thrown at python/sglang/srt/server_args.py:6136

                assert (
                    self._resolved().ep_size == 1
                ), "Triton kernel MoE is only supported when ep_size == 1"

        elif model_arch in ("MiMoV2ForCausalLM", "MiMoV2FlashForCausalLM"):
            if model_arch == "MiMoV2ForCausalLM" and not cfg.encoder_only:
                expected_attn_tp_size = get_mimo_v2_fused_qkv_expected_tp_size(
                    hf_config
                )
                view = self._resolved()
                attn_dp_size = cfg.dp_size if view.enable_dp_attention else 1
                effective_attn_tp_size = (
                    cfg.tp_size // attn_dp_size // view.attn_cp_size
                )
                if (
                    expected_attn_tp_size is not None
                    and expected_attn_tp_size % effective_attn_tp_size != 0
                ):
                    raise ValueError(
                        "MiMoV2ForCausalLM requires effective attention TP "
                        f"size {expected_attn_tp_size} because its fused "
                        "qkv_proj weights are "
                        f"TP={expected_attn_tp_size}-interleaved; got "
                        f"{effective_attn_tp_size} "
                        f"(tp_size={cfg.tp_size}, dp_size={cfg.dp_size}, "
                        f"enable_dp_attention={view.enable_dp_attention}, "
                        f"attn_cp_size={view.attn_cp_size}). "
                        "Set --tp, --dp, --enable-dp-attention, and "
                        "--attention-context-parallel-size so the effective "
                        f"attention TP size is {expected_attn_tp_size}."
                    )

            # enable_multi_layer_eagle for EAGLE moved to the override registry
            # (arg_groups/overrides.py: _mimo_v2_overrides).

            # MiMoV2 hierarchical cache runs on the unified radix tree, which
            # is the default tree cache now. MiMoV2 has head_dim != v_head_dim,

View on GitHub (pinned to 0132848349)

Solutions

1. Reshape the topology so effective attention TP equals the expected factor: drop --enable-dp-attention, reduce --dp-size, or raise --tp-size so tp_size // attn_dp_size // attn_cp_size divides the expected size
2. Check the model config/weights for the qkv TP interleaving factor and match tp_size to it
3. If DP attention is required, increase tp_size so per-rank attention TP stays at the expected degree

Example fix

# before
python -m sglang.launch_server --model MiMoV2 --tp-size 8 --dp-size 8 --enable-dp-attention
# after (effective attn TP = 8 // 8 = 1 → invalid; keep attn TP aligned)
python -m sglang.launch_server --model MiMoV2 --tp-size 8

tp, dp, attn_cp = cfg.tp_size, cfg.dp_size, (cfg.attn_cp_size or 1)
attn_dp = dp if cfg.enable_dp_attention else 1
effective = tp // attn_dp // attn_cp
expected = EXPECTED_MIMO_V2_ATTN_TP  # from model config / fused qkv interleaving
assert expected is None or expected % effective == 0, (
    f"effective attn TP {effective} invalid, need divisor of {expected}")

try:
    server_args = ServerArgs.from_cli_args(cli)
except ValueError as e:
    if "effective attention TP" in str(e):
        raise SystemExit(f"TP/DP topology mismatch for MiMoV2: {e}")
    raise

When it happens

Trigger: Launching MiMoV2 with --tp-size / --dp_size / --enable-dp-attention / attn_cp_size combinations such that tp_size // attn_dp_size // attn_cp_size is not a divisor of expected_attn_tp_size (e.g. dp attention collapsing per-rank attention TP below the fused qkv interleaving).

Common situations: Enabling DP attention or context parallelism on a topology the checkpoint wasn't sharded for; mixing dp_size with tp_size so effective per-rank attention TP shrinks; using a MiMoV2 checkpoint with a different fused-qkv interleaving than a prior launch config.

Related errors

• num_heads ({self.num_heads}) must be divisible by tp_size ({
• Cosmos3CausalAttention requires num_attention_heads divisibl
• Cosmos3CausalAttention requires num_key_value_heads divisibl
• Cosmos3CrossAttention requires num_attention_heads divisible
• Cosmos3CrossAttention requires num_key_value_heads divisible

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/319d761cedf5096a. Report an issue: GitHub.