sgl-project/sglang · error · ValueError

LoRA on Intern-S2-Mobius model.meta_mlp routed banks is not

Error message

LoRA on Intern-S2-Mobius model.meta_mlp routed banks is not supported by the baseline; remove routed-expert targets or use a future bank-aware LoRA implementation.

What it means

Raised in init_lora_modules when LoRA targets gate_up_proj/down_proj (FusedMoE/InklingBatchDenseMLP) on a module whose name cannot be mapped to a decoder layer id and which lives under model.meta_mlp. — the Intern-S2-Mobius routed expert banks. The baseline LoRA implementation only wraps per-layer modules and cannot address these routed banks, so it refuses.

Source

Thrown at python/sglang/srt/lora/lora_manager.py:1002

            if (
                parts[-1] in self.target_modules
                or ".".join(parts[-2:]) in self.target_modules
            ):
                layer_id = get_layer_id(module_name)
                if layer_id is None:
                    continue
                self.lora_modules[layer_id][module_name] = self.set_lora_module(
                    module_name, module
                )
                continue

            if isinstance(module, (FusedMoE, InklingBatchDenseMLP)) and all(
                x in self.target_modules for x in ["gate_up_proj", "down_proj"]
            ):
                layer_id = get_layer_id(module_name)
                if layer_id is None:
                    if module_name.startswith("model.meta_mlp."):
                        raise ValueError(
                            "LoRA on Intern-S2-Mobius model.meta_mlp routed banks "
                            "is not supported by the baseline; remove routed-expert "
                            "targets or use a future bank-aware LoRA implementation."
                        )
                    # FusedMoE submodules outside the decoder layer hierarchy
                    # (e.g. nested helpers under non-".layers." prefixes) have
                    # no resolvable layer id; skip them so we don't index
                    # `self.lora_modules` with `None`.
                    continue
                if isinstance(module, InklingBatchDenseMLP):
                    from sglang.srt.models.inkling_common.lora import (
                        InklingBatchDenseMLPWithLoRA,
                    )

                    module.__class__ = InklingBatchDenseMLPWithLoRA
                    module.initialize_lora(self.lora_backend)
                    lora_module = module
                else:

View on GitHub (pinned to 0132848349)

Solutions

1. Restrict --lora-target-modules to per-layer module suffixes that exclude the meta_mlp routed banks
2. Remove routed-expert targets from the adapter and retrain if you don't need them
3. Wait for / switch to a bank-aware LoRA implementation

Example fix

# before
--lora-target-modules all
# after
--lora-target-modules q_proj k_proj v_proj

banned_context = 'intern-s2-mobius'  # or check model type
if 'gate_up_proj' in server_args.lora_target_modules and 'down_proj' in server_args.lora_target_modules:
    print('restrict --lora-target-modules; routed banks unsupported on this model')

When it happens

Trigger: init_state on an Intern-S2-Mobius style model with --lora-target-modules (or adapter config) including gate_up_proj/down_proj such that model.meta_mlp.* routed-bank submodules match the LoRA wrapper walk.

Common situations: Using --lora-target-modules all or a broad module list on a model with a meta_mlp routed-bank component, unintentionally pulling the non-layer MoE banks into LoRA scope.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Mixed shared-outer LoRA formats detected across loaded adapt
• experimental_sgl_marlin LoRA requires --lora-use-virtual-exp
• Inkling shared-sink LoRA requires four 4D MoE buffers
• Inkling shared-sink LoRA outer factors must have expert dime
• Inkling shared-sink gate-up A and down B must use the same e

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c39f9f6d32503008. Report an issue: GitHub.