{"record":{"id":"8ac6cc53f2cabf46","repo":"sgl-project/sglang","slug":"mixed-shared-outer-lora-formats-detected-across-lo","errorCode":null,"errorMessage":"Mixed shared-outer LoRA formats detected across loaded adapters (conflict in adapter '{adapter_id}'). All MoE adapters must either all use shared outer experts (expert_dim=1) or all use per-expert weights.","messagePattern":"Mixed shared-outer LoRA formats detected across loaded adapters \\(conflict in adapter '(.+?)'\\)\\. All MoE adapters must either all use shared outer experts \\(expert_dim=1\\) or all use per-expert weights\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/lora/lora_manager.py","lineNumber":644,"sourceCode":"        \"\"\"\n        shared_outer: Optional[bool] = None\n        for adapter_id, adapter in self.loras.items():\n            for layer in adapter.layers:\n                for name, weight in layer.weights.items():\n                    if \"gate_up_proj\" not in name or \"lora_A\" not in name:\n                        continue\n                    if weight.dim() == 3:\n                        is_shared = weight.shape[0] == 1\n                    elif re.search(r\"(?:shared_)?experts\\.\\d+\\.\", name):\n                        # Per-expert adapters keep numbered 2D expert weights;\n                        # they must count against the layout agreement too.\n                        is_shared = False\n                    else:\n                        continue\n                    if shared_outer is None:\n                        shared_outer = is_shared\n                    elif shared_outer != is_shared:\n                        raise RuntimeError(\n                            \"Mixed shared-outer LoRA formats detected across \"\n                            f\"loaded adapters (conflict in adapter '{adapter_id}'). \"\n                            \"All MoE adapters must either all use shared outer \"\n                            \"experts (expert_dim=1) or all use per-expert weights.\"\n                        )\n        return bool(shared_outer) if shared_outer is not None else False\n\n    def init_lora_shapes(\n        self,\n        max_lora_rank: Optional[int] = None,\n        target_modules: Optional[Iterable[str]] = None,\n    ):\n        \"\"\"Infer LoRA target modules and max_lora_rank from loaded adapters if not provided.\"\"\"\n\n        if target_modules and target_modules == {\"all\"}:\n            self.target_modules = auto_detect_lora_target_modules(self.base_model)\n            self.target_modules.update(EMBEDDING_NAMES)\n            logger.info(","sourceCodeStart":626,"sourceCodeEnd":662,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/lora/lora_manager.py#L626-L662","documentation":"Raised by _detect_shared_outer_loras when scanning loaded MoE LoRA adapters: it determines whether each adapter uses shared outer experts (expert_dim=1 in the outer weight shape) or per-expert outer weights, and throws if some adapters use one format and others the other. The fused kernels assume a single uniform layout across all adapters sharing the weight buffers.","triggerScenarios":"init_state with multiple MoE LoRA adapters loaded where at least one has outer weights with expert_dim=1 and another has per-expert outer dimensions (e.g. adapters exported by different training/export tool versions).","commonSituations":"Mixing adapters fine-tuned with shared-outer MoE LoRA and adapters trained with per-expert LoRA on the same server; upgrading a training library that changed the export shape.","solutions":["Serve the conflicting adapter(s) on a separate server instance","Re-export/retrain all MoE adapters with the same layout (all shared outer expert_dim=1 or all per-expert)","Load only adapters of one format in a given server run"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# before init_state, check all MoE adapters use one outer format\nshapes = [w.shape for a in adapters for w in a.outer_weights]\nexpert_dims = {s[0] if len(s) == 3 else 1 for s in shapes}  # illustrative\nassert len(expert_dims) == 1, \"mixed shared-outer formats\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Export all MoE adapters with the same tool/version and layout","Group adapters by format and serve each group on its own instance"],"tags":["lora","moe","shape-mismatch","sglang"],"backgroundTag":"incompatible-version-mix","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}