sgl-project/sglang · error · ValueError

LoRA adapter {lora_ref.lora_name} with rank {lora_config.r}

Error message

LoRA adapter {lora_ref.lora_name} with rank {lora_config.r} is incompatible with the current LoRA memory pool configuration. Please ensure that the LoRA adapter's rank is within the configured `--max-lora-rank` and that the target modules are included in `--lora-target-modules`.

What it means

Raised when a new LoRA adapter's configuration (rank, target modules) cannot be supported by the pre-allocated LoRA memory pool. SGLang pre-allocates LoRA buffer sizes at startup from --max-lora-rank and --lora-target-modules, so an adapter whose rank exceeds that or whose target modules fall outside the pool's supported set fails validation via memory_pool.can_support(lora_config).

Source

Thrown at python/sglang/srt/lora/lora_manager.py:306

        # Check if this LoRA adapter is already loaded
        for existing_lora_ref in self.lora_refs.values():
            if lora_ref.lora_name == existing_lora_ref.lora_name:
                raise ValueError(
                    f"Failed to load LoRA adapter {lora_ref.lora_name} because it is already loaded"
                )

            if lora_ref.lora_path == existing_lora_ref.lora_path:
                logger.warning(
                    f"{lora_ref.lora_path} is already loaded with name: {existing_lora_ref.lora_name}, "
                    f"but another copy is being loaded with name: {lora_ref.lora_name}"
                )

        # Check if the LoRA adapter shape is compatible with the current LoRA memory pool configuration.
        memory_pool = getattr(self, "memory_pool", None)
        incompatible = memory_pool and not memory_pool.can_support(lora_config)
        if incompatible:
            raise ValueError(
                f"LoRA adapter {lora_ref.lora_name} with rank {lora_config.r} is incompatible with the current "
                "LoRA memory pool configuration. Please ensure that the LoRA adapter's rank is within the configured "
                "`--max-lora-rank` and that the target modules are included in `--lora-target-modules`."
            )

        # Ensure pinned LoRA adapters does not exceed maximal limit or cause starvation.
        if lora_ref.pinned and self.num_pinned_loras >= self.max_loras_per_batch - 1:
            raise ValueError(
                f"Failed to load LoRA adapter {lora_ref.lora_name} as a pinned adapter. It is not allowed to pin all slots "
                "in the LoRA memory pool to avoid starvation for unpinned adapters and base models. Please increase your "
                "`--max-loras-per-batch` or load it as unpinned LoRA adapters."
            )

    def unload_lora_adapter(self, lora_ref: LoRARef) -> LoRAUpdateOutput:
        logger.info(
            f"LoRA adapter unloading starts: {lora_ref}. "
            f"avail mem={get_available_gpu_memory(self.device.type, self.device.index):.2f} GB"
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Restart the server with --max-lora-rank at least the adapter's rank (e.g. --max-lora-rank 128)
2. Add the adapter's target module suffixes to --lora-target-modules (or use 'all')
3. Retrain/export the adapter with a smaller rank that fits the current pool

Example fix

# before
python -m sglang.launch_server --model ... --enable-lora --max-lora-rank 16
# after
python -m sglang.launch_server --model ... --enable-lora --max-lora-rank 128 --lora-target-modules 'all'

from peft import PeftConfig
cfg = PeftConfig.from_pretrained(path)
assert cfg.r <= server_args.max_lora_rank, f"rank {cfg.r} exceeds --max-lora-rank"
assert set(cfg.target_modules) <= set(server_args.lora_target_modules)

When it happens

Trigger: Loading a LoRA (via load API or startup lora_paths) whose PEFT config rank r > --max-lora-rank, or whose target modules are not covered by the pool built from --lora-target-modules.

Common situations: Trained an adapter with rank 128 but server started with default/lower --max-lora-rank (e.g. 16 or 64); adapter targets q_proj/k_proj but pool was configured only for gate_up_proj/down_proj.

Related errors

• Failed to load LoRA adapter {lora_ref.lora_name} as a pinned
• {selection_error}{component_suffix}
• Cosmos3CausalAttention requires num_attention_heads divisibl
• VLA action expert should not share the prefix TP layout. Use
• PEFT adapter_config.json must contain a JSON object

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e7665b09390eff1b. Report an issue: GitHub.