{"record":{"id":"f98b2b0941954d72","repo":"hiyouga/LlamaFactory","slug":"module-module-name-not-found-in-hidden-modules","errorCode":null,"errorMessage":"Module {module_name} not found in hidden modules: {hidden_modules}","messagePattern":"Module (.+?) not found in hidden modules: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/peft.py","lineNumber":264,"sourceCode":"        if \".0.\" in name:\n            hidden_modules.add(name.split(\".0.\")[-1].split(\".\")[0])\n        elif \".1.\" in name:\n            hidden_modules.add(name.split(\".1.\")[-1].split(\".\")[0])\n\n        if re.search(r\"\\.\\d+\\.\", name) is None:\n            non_hidden_modules.add(name.split(\".\")[-2])\n\n    # Build list of trainable layer patterns\n    trainable_layers = []\n    for module_name in freeze_trainable_modules:\n        if module_name == \"all\":\n            for idx in trainable_layer_ids:\n                trainable_layers.append(f\".{idx:d}.\")\n        elif module_name in hidden_modules:\n            for idx in trainable_layer_ids:\n                trainable_layers.append(f\".{idx:d}.{module_name}\")\n        else:\n            raise ValueError(f\"Module {module_name} not found in hidden modules: {hidden_modules}\")\n\n    # Add extra modules\n    if freeze_extra_modules:\n        for module_name in freeze_extra_modules:\n            if module_name in non_hidden_modules:\n                trainable_layers.append(module_name)\n            else:\n                raise ValueError(f\"Module {module_name} not found in non-hidden modules: {non_hidden_modules}\")\n\n    # TODO\n    # Multi-modal special handling\n\n    # Set requires_grad\n    forbidden_modules = {\"quant_state\", \"quantization_weight\", \"qweight\", \"qzeros\", \"scales\"}\n    for name, param in model.named_parameters():\n        if any(trainable_layer in name for trainable_layer in trainable_layers) and not any(\n            forbidden_module in name for forbidden_module in forbidden_modules\n        ):","sourceCodeStart":246,"sourceCodeEnd":282,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/peft.py#L246-L282","documentation":"During freeze tuning, each name in freeze_trainable_modules is looked up in the set of module-type names found one level before the leaf parameters (e.g. 'q_proj', 'mlp'). If the requested module name never occurs as a parent of any parameter, the plugin cannot build trainable-layer patterns and aborts. This almost always means the target model uses different submodule naming than the default (which assumes qwen/llama-style names like all-linear or q_proj).","triggerScenarios":"freeze_trainable_modules contains a name (e.g. 'attention') that does not match any second-to-last dotted component of model.named_parameters() for the loaded architecture; or the model was replaced but the module list was not updated.","commonSituations":"Copying a freeze config written for Llama/Qwen (q_proj, k_proj, v_proj...) onto a model with different internal names; typos in the module list; using 'all' vs explicit names inconsistently with the architecture.","solutions":["Print {name.split('.')[-2] for name, _ in model.named_parameters()} to see valid module names for your model","Replace freeze_trainable_modules with names that actually appear (e.g. ['q_proj','k_proj','v_proj']) or use 'all'","Fix typos in the YAML module list","For multimodal models, verify the module list targets the LLM backbone names, not vision-tower names"],"exampleFix":"# before\nfreeze_trainable_modules: [\"attention\"]\n\n# after\nfreeze_trainable_modules: [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\"]","handlingStrategy":"validation","validationCode":"def hidden_module_names(model) -> set[str]:\n    return {n.split(\".\")[-2] for n, _ in model.named_parameters()}\n\nwant = [m for m in freeze_trainable_modules if m != \"all\"]\nmissing = [m for m in want if m not in hidden_module_names(model)]\nassert not missing, f\"unknown modules: {missing}; valid: {sorted(hidden_module_names(model))}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive freeze_trainable_modules from model.named_parameters() instead of hardcoding","Use 'all' when unsure of architecture-specific names","Add a config lint step that diffs module lists against the loaded model"],"tags":["peft","freeze-tuning","module-names","configuration"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}