{"record":{"id":"bae879f21e090847","repo":"hiyouga/LlamaFactory","slug":"current-model-does-not-support-freeze-tuning-bae879","errorCode":null,"errorMessage":"Current model does not support freeze tuning.","messagePattern":"Current model does not support freeze tuning\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/peft.py","lineNumber":233,"sourceCode":"    freeze_trainable_modules = peft_config.freeze_trainable_modules\n    freeze_extra_modules = peft_config.freeze_extra_modules\n    cast_trainable_params_to_fp32 = peft_config.cast_trainable_params_to_fp32\n\n    if isinstance(freeze_trainable_modules, str):\n        freeze_trainable_modules = [module.strip() for module in freeze_trainable_modules.split(\",\")]\n\n    if isinstance(freeze_extra_modules, str):\n        freeze_extra_modules = [module.strip() for module in freeze_extra_modules.split(\",\")]\n\n    # Get number of layers\n    num_layers = (\n        getattr(model.config, \"num_hidden_layers\", None)\n        or getattr(model.config, \"num_layers\", None)\n        or getattr(model.config, \"n_layer\", None)\n    )\n\n    if not num_layers:\n        raise ValueError(\"Current model does not support freeze tuning.\")\n\n    if freeze_trainable_layers > 0:\n        # last n layers\n        trainable_layer_ids = range(max(0, num_layers - freeze_trainable_layers), num_layers)\n    else:\n        # first n layers\n        trainable_layer_ids = range(min(-freeze_trainable_layers, num_layers))\n\n    # Identify hidden and non-hidden modules\n    hidden_modules = set()\n    non_hidden_modules = set()\n    for name, _ in model.named_parameters():\n        if \".0.\" in name:\n            hidden_modules.add(name.split(\".0.\")[-1].split(\".\")[0])\n        elif \".1.\" in name:\n            hidden_modules.add(name.split(\".1.\")[-1].split(\".\")[0])\n\n        if re.search(r\"\\.\\d+\\.\", name) is None:","sourceCodeStart":215,"sourceCodeEnd":251,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/peft.py#L215-L251","documentation":"Thrown by the v1 PEFT plugin's freeze-tuning setup when the model config exposes none of the layer-count attributes it probes (num_hidden_layers, num_layers, n_layer). Freeze tuning works by selecting trainable layers by index, so the plugin must know how many decoder layers exist. Architectures that do not report a layer count (some custom or non-transformer models) cannot be freeze-tuned this way.","triggerScenarios":"Setting a freeze peft_config (name 'freeze') on a model whose config lacks all three of num_hidden_layers / num_layers / n_layer, e.g. custom architectures, some Mamba/CNN hybrids, or models loaded with an incomplete config.","commonSituations":"User switches from a Llama/Qwen checkpoint to an exotic or in-house architecture and reuses the same freeze-tuning YAML; or the model was exported with a minimal config.json that omits the layer-count field.","solutions":["Check model.config for a layer-count attribute (print model.config) and confirm the architecture is a stacked decoder/encoder model","If the config uses a non-standard attribute name, patch the config or subclass to expose num_hidden_layers before creating the trainer","Switch to lora or another peft method that does not need layer indices","If you maintain the model code, add num_hidden_layers (or n_layer) to the config class"],"exampleFix":"# before\npeft_config:\n  name: freeze\n  freeze_trainable_layers: 8\n# model: custom architecture without num_hidden_layers\n\n# after\npeft_config:\n  name: lora\n  lora_rank: 8","handlingStrategy":"validation","validationCode":"def supports_freeze_tuning(model) -> bool:\n    cfg = model.config\n    return bool(getattr(cfg, \"num_hidden_layers\", None) or getattr(cfg, \"num_layers\", None) or getattr(cfg, \"n_layer\", None))\n\nif not supports_freeze_tuning(model):\n    raise SystemExit(\"model lacks layer-count attr; use lora instead of freeze\")","typeGuard":null,"tryCatchPattern":"try:\n    trainer = create_trainer(cfg)  # freeze setup\nexcept ValueError as e:\n    if \"freeze tuning\" in str(e):\n        logger.error(\"architecture unsupported for freeze; falling back disabled\")\n    raise","preventionTips":["Before choosing freeze tuning, assert the model config exposes a layer count","Keep per-architecture peft method maps in config templates","Prefer lora for untested architectures"],"tags":["peft","freeze-tuning","model-config","configuration"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}