{"record":{"id":"c41f914b260a9f58","repo":"hiyouga/LlamaFactory","slug":"current-model-does-not-support-freeze-tuning","errorCode":null,"errorMessage":"Current model does not support freeze tuning.","messagePattern":"Current model does not support freeze tuning\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/model/adapter.py","lineNumber":78,"sourceCode":"    is_trainable: bool,\n    cast_trainable_params_to_fp32: bool,\n) -> None:\n    if not is_trainable:\n        return\n\n    logger.info_rank0(\"Fine-tuning method: Freeze\")\n    if hasattr(model.config, \"text_config\"):  # composite models\n        config = getattr(model.config, \"text_config\")\n    else:\n        config = model.config\n\n    num_layers = (\n        getattr(config, \"num_hidden_layers\", None)\n        or getattr(config, \"num_layers\", None)\n        or getattr(config, \"n_layer\", None)\n    )\n    if not num_layers:\n        raise ValueError(\"Current model does not support freeze tuning.\")\n\n    if finetuning_args.use_llama_pro:\n        if num_layers % finetuning_args.freeze_trainable_layers != 0:\n            raise ValueError(\n                f\"`num_layers` {num_layers} should be \"\n                f\"divisible by `num_layer_trainable` {finetuning_args.freeze_trainable_layers}.\"\n            )\n\n        stride = num_layers // finetuning_args.freeze_trainable_layers\n        trainable_layer_ids = range(stride - 1, num_layers + stride - 1, stride)\n    elif finetuning_args.freeze_trainable_layers > 0:  # fine-tuning the last n layers if num_layer_trainable > 0\n        trainable_layer_ids = range(max(0, num_layers - finetuning_args.freeze_trainable_layers), num_layers)\n    else:  # fine-tuning the first n layers if num_layer_trainable < 0\n        trainable_layer_ids = range(min(-finetuning_args.freeze_trainable_layers, num_layers))\n\n    hidden_modules = set()\n    non_hidden_modules = set()\n    for name, _ in model.named_parameters():","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/model/adapter.py#L60-L96","documentation":"Raised in _setup_freeze_tuning when the model config exposes none of num_hidden_layers, num_layers or n_layer, so the number of transformer layers cannot be determined. Freeze tuning works by selecting layer indices, and without a layer count the trainable-layer ranges cannot be computed.","triggerScenarios":"Running with finetuning_type: freeze on an exotic or multimodal architecture whose config uses a different attribute name for the layer count (also after the text_config fallback for composite models fails).","commonSituations":"Freeze-tuning a newly supported or custom-arch model whose config schema LlamaFactory does not recognize; loading a composite model whose text_config also lacks the standard fields.","solutions":["Switch finetuning_type to lora, which does not need the layer count.","Check model.config (and model.config.text_config if present) in a REPL to find the actual layer-count attribute; if the model is one you control, expose num_hidden_layers.","Open/patch adapter.py to read the correct attribute for that architecture."],"exampleFix":"# before\nfinetuning_type: freeze\n\n# after\nfinetuning_type: lora","handlingStrategy":"type-guard","validationCode":"from transformers import AutoConfig\ncfg = AutoConfig.from_pretrained(model_path)\nif hasattr(cfg, \"text_config\"):\n    cfg = cfg.text_config\nlayer_count = getattr(cfg, \"num_hidden_layers\", None) or getattr(cfg, \"num_layers\", None) or getattr(cfg, \"n_layer\", None)\nif finetuning_type == \"freeze\":\n    assert layer_count, \"model config exposes no layer count; freeze tuning unsupported, use lora\"","typeGuard":"def supports_freeze(config) -> bool:\n    cfg = getattr(config, \"text_config\", config)\n    return bool(\n        getattr(cfg, \"num_hidden_layers\", None)\n        or getattr(cfg, \"num_layers\", None)\n        or getattr(cfg, \"n_layer\", None)\n    )","tryCatchPattern":null,"preventionTips":["Probe the config layer count before choosing freeze tuning on a new architecture.","Default new/arch exotic models to lora until freeze compatibility is confirmed."],"tags":["freeze-tuning","model-config","architecture","layer-count"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}