{"record":{"id":"12e102722cb1ddc4","repo":"hiyouga/LlamaFactory","slug":"module-module-name-not-found-in-non-hidden-modul","errorCode":null,"errorMessage":"Module {module_name} not found in non-hidden modules: {non_hidden_modules}","messagePattern":"Module (.+?) not found in non-hidden modules: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/peft.py","lineNumber":272,"sourceCode":"    # Build list of trainable layer patterns\n    trainable_layers = []\n    for module_name in freeze_trainable_modules:\n        if module_name == \"all\":\n            for idx in trainable_layer_ids:\n                trainable_layers.append(f\".{idx:d}.\")\n        elif module_name in hidden_modules:\n            for idx in trainable_layer_ids:\n                trainable_layers.append(f\".{idx:d}.{module_name}\")\n        else:\n            raise ValueError(f\"Module {module_name} not found in hidden modules: {hidden_modules}\")\n\n    # Add extra modules\n    if freeze_extra_modules:\n        for module_name in freeze_extra_modules:\n            if module_name in non_hidden_modules:\n                trainable_layers.append(module_name)\n            else:\n                raise ValueError(f\"Module {module_name} not found in non-hidden modules: {non_hidden_modules}\")\n\n    # TODO\n    # Multi-modal special handling\n\n    # Set requires_grad\n    forbidden_modules = {\"quant_state\", \"quantization_weight\", \"qweight\", \"qzeros\", \"scales\"}\n    for name, param in model.named_parameters():\n        if any(trainable_layer in name for trainable_layer in trainable_layers) and not any(\n            forbidden_module in name for forbidden_module in forbidden_modules\n        ):\n            param.requires_grad_(True)\n            if cast_trainable_params_to_fp32:\n                param.data = param.data.to(torch.float32)  # Cast to fp32 for stability\n        else:\n            param.requires_grad_(False)\n\n    logger.info_rank0(f\"Set trainable layers: {trainable_layers}\")\n","sourceCodeStart":254,"sourceCodeEnd":290,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/peft.py#L254-L290","documentation":"Freeze tuning validates freeze_extra_modules against non_hidden_modules, the set of parameter-parent names that appear OUTSIDE the numbered layer blocks (e.g. 'embed_tokens', 'norm', 'lm_head'). If an extra module name is not found there, no parameter would match it, so the plugin rejects the config. The name must be an existing top-level (non-layer) submodule of the model.","triggerScenarios":"freeze_extra_modules lists a name like 'embeddings' or 'head' that does not occur as the second-to-last component of any parameter outside the decoder layers; or the name exists only inside layers (belongs in freeze_trainable_modules instead).","commonSituations":"Users try to unfreeze embeddings/head with guessed names ('embedding', 'output') instead of the model's actual names (embed_tokens, lm_head); mixing up freeze_trainable_modules and freeze_extra_modules semantics.","solutions":["Inspect non-hidden parameter names: print({n.split('.')[-2] for n,_ in model.named_parameters() if not any(f'.{i}.' in n for i in range(100))})","Use the exact names, typically 'embed_tokens' and 'norm' (and 'lm_head' if untied)","Remove modules that live inside layers from freeze_extra_modules and put them in freeze_trainable_modules"],"exampleFix":"# before\nfreeze_extra_modules: [\"embedding\", \"head\"]\n\n# after\nfreeze_extra_modules: [\"embed_tokens\", \"norm\", \"lm_head\"]","handlingStrategy":"validation","validationCode":"import re\nnon_hidden = {n.split(\".\")[-2] for n, _ in model.named_parameters() if not re.search(r\"\\.\\d+\\.\", n)}\nmissing = [m for m in freeze_extra_modules if m not in non_hidden]\nassert not missing, f\"extra modules not found: {missing}; valid: {sorted(non_hidden)}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Discover embed/head names from the actual checkpoint before writing them into YAML","Keep layer-internal modules in freeze_trainable_modules and top-level modules in freeze_extra_modules","Automate config generation from a model-introspection script"],"tags":["peft","freeze-tuning","module-names","configuration"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}