{"record":{"id":"f76a2ad99e576a69","repo":"hiyouga/LlamaFactory","slug":"num-layers-num-layers-should-be-divisible-by","errorCode":null,"errorMessage":"`num_layers` {num_layers} should be divisible by `num_expand` {num_expand}.","messagePattern":"`num_layers` (.+?) should be divisible by `num_expand` (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/llama_pro.py","lineNumber":54,"sourceCode":"def change_name(name: str, old_index: int, new_index: int) -> str:\n    return name.replace(f\".{old_index:d}.\", f\".{new_index:d}.\")\n\n\ndef block_expansion(\n    model_name_or_path: str,\n    output_dir: str,\n    num_expand: int,\n    shard_size: str = \"5GB\",\n    save_safetensors: bool = True,\n):\n    r\"\"\"Perform block expansion for LLaMA, Mistral, Qwen2 or Yi models.\n\n    Usage: python llama_pro.py --model_name_or_path meta-llama/Llama-2-7b-hf --output_dir llama2_pro --num_expand 8\n    \"\"\"\n    config: PretrainedConfig = AutoConfig.from_pretrained(model_name_or_path, trust_remote_code=True)\n    num_layers = getattr(config, \"num_hidden_layers\")\n    if num_layers % num_expand != 0:\n        raise ValueError(f\"`num_layers` {num_layers} should be divisible by `num_expand` {num_expand}.\")\n\n    setattr(config, \"num_hidden_layers\", num_layers + num_expand)\n    config.save_pretrained(output_dir)\n\n    tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)\n    tokenizer.save_pretrained(output_dir)\n\n    print(f\"Expanding model of {num_layers} layers to {num_layers + num_expand} layers.\")\n    model = AutoModelForCausalLM.from_pretrained(\n        model_name_or_path, torch_dtype=\"auto\", device_map=\"cpu\", trust_remote_code=True, low_cpu_mem_usage=True\n    )\n    assert isinstance(model, PreTrainedModel)  # type hint\n    if save_safetensors and getattr(model.config, \"tie_word_embeddings\", False):\n        del model.lm_head  # safetensors does not allow shared weights\n\n    split = num_layers // num_expand\n    layer_cnt = 0\n    state_dict = model.state_dict()","sourceCodeStart":36,"sourceCodeEnd":72,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/scripts/llama_pro.py#L36-L72","documentation":"After calling Trainer.__init__, the DPO trainer requires self.accelerator to exist (trainer.py:90). self.accelerator is set by modern transformers Trainers; if the installed transformers is too old (pre-accelerate-integration or a version where the attribute is created later/differently), hasattr fails and the trainer aborts with AttributeError instead of crashing obscurely mid-training.","triggerScenarios":"Running DPO/KTO-style training (run_dpo) with an outdated transformers version; Trainer.__init__ completes but never assigns self.accelerator, so the guard raises immediately after super().__init__.","commonSituations":"Pinning transformers < 4.x-latest for another model while running DPO; environments where another package downgraded transformers; stale conda envs.","solutions":["pip install -U transformers (use the version pinned in LlamaFactory's requirements.txt / pyproject for a known-good pair)","Verify with `python -c \"import transformers; print(transformers.__version__)\"` after upgrading to confirm no other tool downgraded it"],"exampleFix":"# before\ntransformers 4.3x in env -> AttributeError: Please update `transformers`.\n\n# after\npip install -U transformers\npython -c \"import transformers; print(transformers.__version__)\"","handlingStrategy":"validation","validationCode":"import transformers\nfrom packaging.version import parse\nv = parse(transformers.__version__)\nassert not parse('4.30') <= v < parse('4.37'), (\n    f'transformers {v} too old for DPO trainer; pip install -U transformers'\n)","typeGuard":"def transformers_new_enough_for_dpo() -> bool:\n    import transformers\n    from packaging.version import parse\n    return parse(transformers.__version__) >= parse('4.37.0')","tryCatchPattern":"try:\n    from llamafactory.train.dpo.workflow import run_dpo\n    run_dpo(train_args)\nexcept AttributeError as e:\n    if 'update `transformers`' in str(e):\n        raise SystemExit('pip install -U transformers') from e\n    raise","preventionTips":["Freeze transformers to LlamaFactory's pinned version in every training image","Print the versions of transformers/accelerate/trl at job start for reproducible debugging"],"tags":["version","transformers","dpo","dependency"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}