hiyouga/LlamaFactory · error · ValueError

Currently lora stage does not support loading model by meta.

Error message

Currently lora stage does not support loading model by meta.

What it means

Raised by the v1 ModelEngine when a LoRA adapter setup is combined with init_mode == 'init_on_meta'. The engine can materialize the base model on the meta device for full tuning, but the PeftPlugin path needs real module instances to wrap with LoRA layers, so meta-device initialization is explicitly rejected. It is a hard stop before any weights are touched.

Source

Thrown at src/llamafactory/v1/core/model_engine.py:212

                **init_kwargs,
            )

        init_mode = self.args.init_config.name if self.args.init_config is not None else "init_on_default"
        model._init_mode = init_mode

        if hasattr(model, "thinker"):
            model = model.thinker
            model._init_mode = init_mode

        if self.args.peft_config is None:
            if self.is_train:
                logger.info_rank0("Fine-tuning mode: full tuning")
                model = model.to(torch.float32)
            else:
                logger.info_rank0("Inference the original model")
        else:
            if self.args.peft_config.name == "lora" and init_mode == "init_on_meta":
                raise ValueError("Currently lora stage does not support loading model by meta.")

            from ..plugins.model_plugins.peft import PeftPlugin

            model = PeftPlugin(self.args.peft_config.name)(
                model,
                peft_config=self.args.peft_config,
                is_train=self.is_train,
            )

        if self.args.kernel_config is not None:
            from ..plugins.model_plugins.kernels.interface import apply_kernels

            model = apply_kernels(model, self.args.kernel_config, require_logits=self.is_train)

        return model


if __name__ == "__main__":

View on GitHub (pinned to f28afaf635)

Solutions

1. Change init_mode away from 'init_on_meta' (e.g. normal eager/deferred weight loading) so the LoRA plugin can wrap concrete modules
2. Or switch from LoRA to full fine-tuning (drop peft_config), which does support meta init in this engine
3. Or stay on the v0 path (do not set USE_V1=1) where LoRA + meta loading combinations may be handled differently

Example fix

# before
engine = ModelEngine(args_with(peft_config=lora_cfg), init_mode="init_on_meta")  # ValueError

# after
engine = ModelEngine(args_with(peft_config=lora_cfg), init_mode="deferred")  # concrete modules for PeftPlugin

def check_lora_init_mode(peft_config, init_mode: str) -> None:
    if peft_config is not None and peft_config.name == "lora" and init_mode == "init_on_meta":
        raise SystemExit("LoRA cannot use init_on_meta; pick another init_mode or full tuning")

When it happens

Trigger: Constructing the v1 model engine with args.peft_config set (peft_config.name == 'lora') while the model is created with init_mode='init_on_meta' (e.g. a trainer/kernel plugin that requests meta-device init to save CPU memory).

Common situations: Enabling a memory-saving meta-device/kernel config (kernel_config, FSDP2-style delayed init) together with a LoRA finetune; porting a v0 workflow that loaded models on meta into the experimental v1 architecture.

Related errors

• When `adapter_name_or_path` is provided for training, only a
• Currently merge and export model function is only supported
• Please set adapter_name_or_path to merge adapters into base
• The current model does not support `chat`.
• The current model does not support `stream_chat`.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/dabe41f4e3819c4b. Report an issue: GitHub.