{"record":{"id":"319d761cedf5096a","repo":"sgl-project/sglang","slug":"mimov2forcausallm-requires-effective-attention-tp","errorCode":null,"errorMessage":"MiMoV2ForCausalLM requires effective attention TP size {expected_attn_tp_size} because its fused qkv_proj weights are TP={expected_attn_tp_size}-interleaved; got {effective_attn_tp_size} (tp_size={cfg.tp_size}, dp_size={cfg.dp_size}, enable_dp_attention={view.enable_dp_attention}, attn_cp_size={view.attn_cp_size}). ","messagePattern":"MiMoV2ForCausalLM requires effective attention TP size (.+?) because its fused qkv_proj weights are TP=(.+?)-interleaved; got (.+?) \\(tp_size=(.+?), dp_size=(.+?), enable_dp_attention=(.+?), attn_cp_size=(.+?)\\)\\. ","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/server_args.py","lineNumber":6136,"sourceCode":"                assert (\n                    self._resolved().ep_size == 1\n                ), \"Triton kernel MoE is only supported when ep_size == 1\"\n\n        elif model_arch in (\"MiMoV2ForCausalLM\", \"MiMoV2FlashForCausalLM\"):\n            if model_arch == \"MiMoV2ForCausalLM\" and not cfg.encoder_only:\n                expected_attn_tp_size = get_mimo_v2_fused_qkv_expected_tp_size(\n                    hf_config\n                )\n                view = self._resolved()\n                attn_dp_size = cfg.dp_size if view.enable_dp_attention else 1\n                effective_attn_tp_size = (\n                    cfg.tp_size // attn_dp_size // view.attn_cp_size\n                )\n                if (\n                    expected_attn_tp_size is not None\n                    and expected_attn_tp_size % effective_attn_tp_size != 0\n                ):\n                    raise ValueError(\n                        \"MiMoV2ForCausalLM requires effective attention TP \"\n                        f\"size {expected_attn_tp_size} because its fused \"\n                        \"qkv_proj weights are \"\n                        f\"TP={expected_attn_tp_size}-interleaved; got \"\n                        f\"{effective_attn_tp_size} \"\n                        f\"(tp_size={cfg.tp_size}, dp_size={cfg.dp_size}, \"\n                        f\"enable_dp_attention={view.enable_dp_attention}, \"\n                        f\"attn_cp_size={view.attn_cp_size}). \"\n                        \"Set --tp, --dp, --enable-dp-attention, and \"\n                        \"--attention-context-parallel-size so the effective \"\n                        f\"attention TP size is {expected_attn_tp_size}.\"\n                    )\n\n            # enable_multi_layer_eagle for EAGLE moved to the override registry\n            # (arg_groups/overrides.py: _mimo_v2_overrides).\n\n            # MiMoV2 hierarchical cache runs on the unified radix tree, which\n            # is the default tree cache now. MiMoV2 has head_dim != v_head_dim,","sourceCodeStart":6118,"sourceCodeEnd":6154,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6118-L6154","documentation":"MiMoV2ForCausalLM ships its fused qkv_proj weights pre-sharded/interleaved for a fixed attention tensor-parallel degree. SGLang computes the effective attention TP size as tp_size // attn_dp_size // attn_cp_size and rejects any value that does not evenly divide the weight's expected interleaving factor, since loading would produce corrupt attention projections.","triggerScenarios":"Launching MiMoV2 with --tp-size / --dp_size / --enable-dp-attention / attn_cp_size combinations such that tp_size // attn_dp_size // attn_cp_size is not a divisor of expected_attn_tp_size (e.g. dp attention collapsing per-rank attention TP below the fused qkv interleaving).","commonSituations":"Enabling DP attention or context parallelism on a topology the checkpoint wasn't sharded for; mixing dp_size with tp_size so effective per-rank attention TP shrinks; using a MiMoV2 checkpoint with a different fused-qkv interleaving than a prior launch config.","solutions":["Reshape the topology so effective attention TP equals the expected factor: drop --enable-dp-attention, reduce --dp-size, or raise --tp-size so tp_size // attn_dp_size // attn_cp_size divides the expected size","Check the model config/weights for the qkv TP interleaving factor and match tp_size to it","If DP attention is required, increase tp_size so per-rank attention TP stays at the expected degree"],"exampleFix":"# before\npython -m sglang.launch_server --model MiMoV2 --tp-size 8 --dp-size 8 --enable-dp-attention\n# after (effective attn TP = 8 // 8 = 1 → invalid; keep attn TP aligned)\npython -m sglang.launch_server --model MiMoV2 --tp-size 8","handlingStrategy":"validation","validationCode":"tp, dp, attn_cp = cfg.tp_size, cfg.dp_size, (cfg.attn_cp_size or 1)\nattn_dp = dp if cfg.enable_dp_attention else 1\neffective = tp // attn_dp // attn_cp\nexpected = EXPECTED_MIMO_V2_ATTN_TP  # from model config / fused qkv interleaving\nassert expected is None or expected % effective == 0, (\n    f\"effective attn TP {effective} invalid, need divisor of {expected}\")","typeGuard":null,"tryCatchPattern":"try:\n    server_args = ServerArgs.from_cli_args(cli)\nexcept ValueError as e:\n    if \"effective attention TP\" in str(e):\n        raise SystemExit(f\"TP/DP topology mismatch for MiMoV2: {e}\")\n    raise","preventionTips":["Compute tp//dp//cp per rank before launching hybrid models with fused qkv weights","Keep a per-model matrix of validated tp_size/dp_size/enable_dp_attention combos","Never enable dp_attention for checkpoints with fixed attention interleaving without checking the model card"],"tags":["sglang","tensor-parallel","model-config","dp-attention","mimo"],"backgroundTag":"tensor-parallel-degree-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}