sgl-project/sglang · error · RuntimeError

Mixed shared-outer LoRA formats detected across loaded adapt

Error message

Mixed shared-outer LoRA formats detected across loaded adapters (conflict in adapter '{adapter_id}'). All MoE adapters must either all use shared outer experts (expert_dim=1) or all use per-expert weights.

What it means

Raised by _detect_shared_outer_loras when scanning loaded MoE LoRA adapters: it determines whether each adapter uses shared outer experts (expert_dim=1 in the outer weight shape) or per-expert outer weights, and throws if some adapters use one format and others the other. The fused kernels assume a single uniform layout across all adapters sharing the weight buffers.

Source

Thrown at python/sglang/srt/lora/lora_manager.py:644

        """
        shared_outer: Optional[bool] = None
        for adapter_id, adapter in self.loras.items():
            for layer in adapter.layers:
                for name, weight in layer.weights.items():
                    if "gate_up_proj" not in name or "lora_A" not in name:
                        continue
                    if weight.dim() == 3:
                        is_shared = weight.shape[0] == 1
                    elif re.search(r"(?:shared_)?experts\.\d+\.", name):
                        # Per-expert adapters keep numbered 2D expert weights;
                        # they must count against the layout agreement too.
                        is_shared = False
                    else:
                        continue
                    if shared_outer is None:
                        shared_outer = is_shared
                    elif shared_outer != is_shared:
                        raise RuntimeError(
                            "Mixed shared-outer LoRA formats detected across "
                            f"loaded adapters (conflict in adapter '{adapter_id}'). "
                            "All MoE adapters must either all use shared outer "
                            "experts (expert_dim=1) or all use per-expert weights."
                        )
        return bool(shared_outer) if shared_outer is not None else False

    def init_lora_shapes(
        self,
        max_lora_rank: Optional[int] = None,
        target_modules: Optional[Iterable[str]] = None,
    ):
        """Infer LoRA target modules and max_lora_rank from loaded adapters if not provided."""

        if target_modules and target_modules == {"all"}:
            self.target_modules = auto_detect_lora_target_modules(self.base_model)
            self.target_modules.update(EMBEDDING_NAMES)
            logger.info(

View on GitHub (pinned to 0132848349)

Solutions

1. Serve the conflicting adapter(s) on a separate server instance
2. Re-export/retrain all MoE adapters with the same layout (all shared outer expert_dim=1 or all per-expert)
3. Load only adapters of one format in a given server run

# before init_state, check all MoE adapters use one outer format
shapes = [w.shape for a in adapters for w in a.outer_weights]
expert_dims = {s[0] if len(s) == 3 else 1 for s in shapes}  # illustrative
assert len(expert_dims) == 1, "mixed shared-outer formats"

When it happens

Trigger: init_state with multiple MoE LoRA adapters loaded where at least one has outer weights with expert_dim=1 and another has per-expert outer dimensions (e.g. adapters exported by different training/export tool versions).

Common situations: Mixing adapters fine-tuned with shared-outer MoE LoRA and adapters trained with per-expert LoRA on the same server; upgrading a training library that changed the export shape.

Related errors

• LoRA on Intern-S2-Mobius model.meta_mlp routed banks is not
• experimental_sgl_marlin LoRA requires --lora-use-virtual-exp
• LoRA pinned weight cache key collision for {cache_key!r}: ca
• Inkling shared-sink LoRA requires four 4D MoE buffers
• Inkling shared-sink LoRA outer factors must have expert dime

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8ac6cc53f2cabf46. Report an issue: GitHub.