What causes "Module {} is not found, please choose from {}" in hiyouga/LlamaFactory?

Inspect the error's listed valid choices (it prints the discovered hidden_modules set) and use one of those exact names.

hiyouga/LlamaFactory · error · ValueError

Module {} is not found, please choose from {}

Error message

Module {} is not found, please choose from {}

What it means

Raised in _setup_freeze_tuning when an entry of freeze_trainable_modules (other than the literal 'all') does not appear in hidden_modules, the set of module suffixes discovered inside the model's indexed layers (parsed from parameter names around '.0.' / '.1.'). The configured module names must match the model's real inner-module naming.

Source

Thrown at src/llamafactory/model/adapter.py:108

        trainable_layer_ids = range(max(0, num_layers - finetuning_args.freeze_trainable_layers), num_layers)
    else:  # fine-tuning the first n layers if num_layer_trainable < 0
        trainable_layer_ids = range(min(-finetuning_args.freeze_trainable_layers, num_layers))

    hidden_modules = set()
    non_hidden_modules = set()
    for name, _ in model.named_parameters():
        if ".0." in name:
            hidden_modules.add(name.split(".0.")[-1].split(".")[0])
        elif ".1." in name:  # MoD starts from layer 1
            hidden_modules.add(name.split(".1.")[-1].split(".")[0])

        if re.search(r"\.\d+\.", name) is None:
            non_hidden_modules.add(name.split(".")[-2])  # remove weight/bias

    trainable_layers = []
    for module_name in finetuning_args.freeze_trainable_modules:
        if module_name != "all" and module_name not in hidden_modules:
            raise ValueError(
                "Module {} is not found, please choose from {}".format(module_name, ", ".join(hidden_modules))
            )

        for idx in trainable_layer_ids:
            trainable_layers.append(".{:d}.{}".format(idx, module_name if module_name != "all" else ""))

    if finetuning_args.freeze_extra_modules:
        for module_name in finetuning_args.freeze_extra_modules:
            if module_name not in non_hidden_modules:
                raise ValueError(
                    "Module {} is not found, please choose from {}".format(module_name, ", ".join(non_hidden_modules))
                )

            trainable_layers.append(module_name)

    model_type = getattr(model.config, "model_type", None)
    if not finetuning_args.freeze_multi_modal_projector and model_type in COMPOSITE_MODELS:
        trainable_layers.extend(COMPOSITE_MODELS[model_type].projector_keys)

View on GitHub (pinned to f28afaf635)

Solutions

Inspect the error's listed valid choices (it prints the discovered hidden_modules set) and use one of those exact names.
Use freeze_trainable_modules: all to train every module inside the selected layers instead of a specific one.
Print parameter names via model.named_parameters() to confirm the real module naming for your architecture.

Example fix

# before
freeze_trainable_modules: mlp

# after
freeze_trainable_modules: all

Defensive patterns

Strategy: validation

Validate before calling

import re
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(model_path)
hidden = set()
for name, _ in model.named_parameters():
    if ".0." in name:
        hidden.add(name.split(".0.")[-1].split(".")[0])
    elif ".1." in name:
        hidden.add(name.split(".1.")[-1].split(".")[0])
for m in freeze_trainable_modules:
    if m != "all":
        assert m in hidden, f"{m!r} not in model's layer modules: {sorted(hidden)}"

Prevention

Derive freeze_trainable_modules from the model itself (inspect named_parameters) instead of copying between architectures.
Use 'all' when unsure; it is always valid.

When it happens

Trigger: Setting freeze_trainable_modules: [mlp] on an architecture whose layers do not contain a module literally named mlp (e.g. some models use mlp.c_proj style paths or different names), so the parsed hidden module set never contains it.

Common situations: Copying freeze module lists between architectures (e.g. from Llama to Qwen/GLM/custom models); typos in module names; models with fused or differently named submodules.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/f03e9f38c51ef3de. Report an issue: GitHub.