hiyouga/LlamaFactory · error · ValueError

Module {module_name} not found in hidden modules: {hidden_mo

Error message

Module {module_name} not found in hidden modules: {hidden_modules}

What it means

During freeze tuning, each name in freeze_trainable_modules is looked up in the set of module-type names found one level before the leaf parameters (e.g. 'q_proj', 'mlp'). If the requested module name never occurs as a parent of any parameter, the plugin cannot build trainable-layer patterns and aborts. This almost always means the target model uses different submodule naming than the default (which assumes qwen/llama-style names like all-linear or q_proj).

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/peft.py:264

        if ".0." in name:
            hidden_modules.add(name.split(".0.")[-1].split(".")[0])
        elif ".1." in name:
            hidden_modules.add(name.split(".1.")[-1].split(".")[0])

        if re.search(r"\.\d+\.", name) is None:
            non_hidden_modules.add(name.split(".")[-2])

    # Build list of trainable layer patterns
    trainable_layers = []
    for module_name in freeze_trainable_modules:
        if module_name == "all":
            for idx in trainable_layer_ids:
                trainable_layers.append(f".{idx:d}.")
        elif module_name in hidden_modules:
            for idx in trainable_layer_ids:
                trainable_layers.append(f".{idx:d}.{module_name}")
        else:
            raise ValueError(f"Module {module_name} not found in hidden modules: {hidden_modules}")

    # Add extra modules
    if freeze_extra_modules:
        for module_name in freeze_extra_modules:
            if module_name in non_hidden_modules:
                trainable_layers.append(module_name)
            else:
                raise ValueError(f"Module {module_name} not found in non-hidden modules: {non_hidden_modules}")

    # TODO
    # Multi-modal special handling

    # Set requires_grad
    forbidden_modules = {"quant_state", "quantization_weight", "qweight", "qzeros", "scales"}
    for name, param in model.named_parameters():
        if any(trainable_layer in name for trainable_layer in trainable_layers) and not any(
            forbidden_module in name for forbidden_module in forbidden_modules
        ):

View on GitHub (pinned to f28afaf635)

Solutions

1. Print {name.split('.')[-2] for name, _ in model.named_parameters()} to see valid module names for your model
2. Replace freeze_trainable_modules with names that actually appear (e.g. ['q_proj','k_proj','v_proj']) or use 'all'
3. Fix typos in the YAML module list
4. For multimodal models, verify the module list targets the LLM backbone names, not vision-tower names

Example fix

# before
freeze_trainable_modules: ["attention"]

# after
freeze_trainable_modules: ["q_proj", "k_proj", "v_proj", "o_proj"]

def hidden_module_names(model) -> set[str]:
    return {n.split(".")[-2] for n, _ in model.named_parameters()}

want = [m for m in freeze_trainable_modules if m != "all"]
missing = [m for m in want if m not in hidden_module_names(model)]
assert not missing, f"unknown modules: {missing}; valid: {sorted(hidden_module_names(model))}"

When it happens

Trigger: freeze_trainable_modules contains a name (e.g. 'attention') that does not match any second-to-last dotted component of model.named_parameters() for the loaded architecture; or the model was replaced but the module list was not updated.

Common situations: Copying a freeze config written for Llama/Qwen (q_proj, k_proj, v_proj...) onto a model with different internal names; typos in the module list; using 'all' vs explicit names inconsistently with the architecture.

Related errors

• Module {module_name} not found in non-hidden modules: {non_h
• Current model does not support freeze tuning.
• Module {} is not found, please choose from {}
• When `adapter_name_or_path` is provided for training, only a
• Please specify peft_config to merge and export model.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/f98b2b0941954d72. Report an issue: GitHub.