{"record":{"id":"794dda28bacb9ec7","repo":"hiyouga/LlamaFactory","slug":"model-was-not-supported","errorCode":null,"errorMessage":"Model was not supported.","messagePattern":"Model was not supported\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/model/model_utils/misc.py","lineNumber":59,"sourceCode":"        forbidden_modules.update(COMPOSITE_MODELS[model_type].vision_model_keys)\n\n    module_names = set()\n    for name, module in model.named_modules():\n        if any(forbidden_module in name for forbidden_module in forbidden_modules):\n            continue\n\n        if \"Linear\" in module.__class__.__name__ and \"Embedding\" not in module.__class__.__name__:\n            module_names.add(name.split(\".\")[-1])\n\n    logger.info_rank0(\"Found linear modules: {}\".format(\",\".join(module_names)))\n    return list(module_names)\n\n\ndef find_expanded_modules(model: \"PreTrainedModel\", target_modules: list[str], num_layer_trainable: int) -> list[str]:\n    r\"\"\"Find the modules in the expanded blocks to apply lora.\"\"\"\n    num_layers = getattr(model.config, \"num_hidden_layers\", None)\n    if not num_layers:\n        raise ValueError(\"Model was not supported.\")\n\n    if num_layers % num_layer_trainable != 0:\n        raise ValueError(\n            f\"`num_layers` {num_layers} should be divisible by `num_layer_trainable` {num_layer_trainable}.\"\n        )\n\n    stride = num_layers // num_layer_trainable\n    trainable_layer_ids = range(stride - 1, num_layers + stride - 1, stride)\n    trainable_layers = [f\".{idx:d}.\" for idx in trainable_layer_ids]\n    module_names = []\n    for name, _ in model.named_modules():\n        if any(target_module in name for target_module in target_modules) and any(\n            trainable_layer in name for trainable_layer in trainable_layers\n        ):\n            module_names.append(name)\n\n    logger.info_rank0(\"Apply lora to layers: {}.\".format(\",\".join(map(str, trainable_layer_ids))))\n    return module_names","sourceCodeStart":41,"sourceCodeEnd":77,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/model/model_utils/misc.py#L41-L77","documentation":"find_expanded_modules (misc.py) maps num_layer_trainable onto the model by reading config.num_hidden_layers. If the config lacks num_hidden_layers (getattr returns None), LlamaFactory cannot compute per-block LoRA target layers for the expanded-blocks training mode and raises the generic ValueError 'Model was not supported.'.","triggerScenarios":"Setting finetuning_args.num_layer_trainable (train only every N-th block's LoRA) on a model whose config does not define num_hidden_layers — e.g. some multimodal models that store depth under vision_config/text_config or num_layers, or custom configs with different key names.","commonSituations":"Using num_layer_trainable with a new/custom architecture; models where the layer count lives in a nested config; experimental models added without a patcher entry exposing num_hidden_layers.","solutions":["Do not use num_layer_trainable for this model; rely on standard lora_target over all layers instead.","If it is your model, expose num_hidden_layers on the top-level config (copy from text_config) before loading.","Check config.to_dict() for where layer count lives and file/patch support for that model_type.","Switch to a supported model family when you need the expanded-blocks training mode."],"exampleFix":"# before\nfinetuning_args.num_layer_trainable = 4  # config lacks num_hidden_layers -> ValueError\n\n# after (custom model fix)\nconfig.num_hidden_layers = config.text_config.num_hidden_layers\n# or drop num_layer_trainable and use plain lora_target","handlingStrategy":"validation","validationCode":"num_layers = getattr(model.config, \"num_hidden_layers\", None)\nif finetuning_args.num_layer_trainable > 0:\n    assert num_layers, \"config.num_hidden_layers missing; num_layer_trainable unsupported for this model\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Inspect model.config.to_dict() for the layer-count key before using num_layer_trainable.","Keep a whitelist of model types validated for the expanded-blocks mode."],"tags":["lora","num-layer-trainable","config","custom-model"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}