hiyouga/LlamaFactory · error · ValueError

Current model does not support freeze tuning.

Error message

Current model does not support freeze tuning.

What it means

Raised in _setup_freeze_tuning when the model config exposes none of num_hidden_layers, num_layers or n_layer, so the number of transformer layers cannot be determined. Freeze tuning works by selecting layer indices, and without a layer count the trainable-layer ranges cannot be computed.

Source

Thrown at src/llamafactory/model/adapter.py:78

    is_trainable: bool,
    cast_trainable_params_to_fp32: bool,
) -> None:
    if not is_trainable:
        return

    logger.info_rank0("Fine-tuning method: Freeze")
    if hasattr(model.config, "text_config"):  # composite models
        config = getattr(model.config, "text_config")
    else:
        config = model.config

    num_layers = (
        getattr(config, "num_hidden_layers", None)
        or getattr(config, "num_layers", None)
        or getattr(config, "n_layer", None)
    )
    if not num_layers:
        raise ValueError("Current model does not support freeze tuning.")

    if finetuning_args.use_llama_pro:
        if num_layers % finetuning_args.freeze_trainable_layers != 0:
            raise ValueError(
                f"`num_layers` {num_layers} should be "
                f"divisible by `num_layer_trainable` {finetuning_args.freeze_trainable_layers}."
            )

        stride = num_layers // finetuning_args.freeze_trainable_layers
        trainable_layer_ids = range(stride - 1, num_layers + stride - 1, stride)
    elif finetuning_args.freeze_trainable_layers > 0:  # fine-tuning the last n layers if num_layer_trainable > 0
        trainable_layer_ids = range(max(0, num_layers - finetuning_args.freeze_trainable_layers), num_layers)
    else:  # fine-tuning the first n layers if num_layer_trainable < 0
        trainable_layer_ids = range(min(-finetuning_args.freeze_trainable_layers, num_layers))

    hidden_modules = set()
    non_hidden_modules = set()
    for name, _ in model.named_parameters():

View on GitHub (pinned to f28afaf635)

Solutions

1. Switch finetuning_type to lora, which does not need the layer count.
2. Check model.config (and model.config.text_config if present) in a REPL to find the actual layer-count attribute; if the model is one you control, expose num_hidden_layers.
3. Open/patch adapter.py to read the correct attribute for that architecture.

Example fix

# before
finetuning_type: freeze

# after
finetuning_type: lora

from transformers import AutoConfig
cfg = AutoConfig.from_pretrained(model_path)
if hasattr(cfg, "text_config"):
    cfg = cfg.text_config
layer_count = getattr(cfg, "num_hidden_layers", None) or getattr(cfg, "num_layers", None) or getattr(cfg, "n_layer", None)
if finetuning_type == "freeze":
    assert layer_count, "model config exposes no layer count; freeze tuning unsupported, use lora"

def supports_freeze(config) -> bool:
    cfg = getattr(config, "text_config", config)
    return bool(
        getattr(cfg, "num_hidden_layers", None)
        or getattr(cfg, "num_layers", None)
        or getattr(cfg, "n_layer", None)
    )

When it happens

Trigger: Running with finetuning_type: freeze on an exotic or multimodal architecture whose config uses a different attribute name for the layer count (also after the text_config fallback for composite models fails).

Common situations: Freeze-tuning a newly supported or custom-arch model whose config schema LlamaFactory does not recognize; loading a composite model whose text_config also lacks the standard fields.

Related errors

• `num_layers` {num_layers} should be divisible by `num_layer_
• Module {} is not found, please choose from {}
• Current model does not support freeze tuning.
• Current model is not supported by mixture-of-depth.
• Module {module_name} not found in hidden modules: {hidden_mo

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/c41f914b260a9f58. Report an issue: GitHub.