{"record":{"id":"46a71965356f2143","repo":"hiyouga/LlamaFactory","slug":"cannot-resize-embedding-layers-of-a-quantized-mode","errorCode":null,"errorMessage":"Cannot resize embedding layers of a quantized model.","messagePattern":"Cannot resize embedding layers of a quantized model\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":243,"sourceCode":"\n\ndef _verify_model_args(\n    model_args: \"ModelArguments\",\n    data_args: \"DataArguments\",\n    finetuning_args: \"FinetuningArguments\",\n) -> None:\n    if model_args.adapter_name_or_path is not None and finetuning_args.finetuning_type != \"lora\":\n        raise ValueError(\"Adapter is only valid for the LoRA method.\")\n\n    if model_args.quantization_bit is not None:\n        if finetuning_args.finetuning_type not in [\"lora\", \"oft\"]:\n            raise ValueError(\"Quantization is only compatible with the LoRA or OFT method.\")\n\n        if finetuning_args.pissa_init:\n            raise ValueError(\"Please use scripts/pissa_init.py to initialize PiSSA for a quantized model.\")\n\n        if model_args.resize_vocab:\n            raise ValueError(\"Cannot resize embedding layers of a quantized model.\")\n\n        if model_args.adapter_name_or_path is not None and finetuning_args.create_new_adapter:\n            raise ValueError(\"Cannot create new adapter upon a quantized model.\")\n\n        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:\n            raise ValueError(\"Quantized model only accepts a single adapter. Merge them first.\")\n\n\ndef _check_extra_dependencies(\n    model_args: \"ModelArguments\",\n    finetuning_args: \"FinetuningArguments\",\n    training_args: Optional[\"TrainingArguments\"] = None,\n) -> None:\n    if model_args.use_kt:\n        check_version(\"kt-kernel\", mandatory=True)\n        check_version(\"transformers-kt\", mandatory=True)\n        check_version(\"accelerate-kt\", mandatory=True)\n","sourceCodeStart":225,"sourceCodeEnd":261,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L225-L261","documentation":"Raised by _verify_model_args when quantization_bit is set and resize_vocab is true. Resizing embedding layers rewrites the token-embedding matrix with new randomly initialized rows; on a quantized model those weights are stored in compressed INT4/INT8 form and cannot be resized in place.","triggerScenarios":"A YAML combining quantization_bit: 4/8 with resize_vocab: true (typically added to support added special tokens after training on a bigger tokenizer); _verify_model_args rejects it before loading.","commonSituations":"Users fine-tune with additional tokens (chat templates, tool markers) and keep QLoRA quantization on, expecting vocab resizing to work as in BF16 runs.","solutions":["Set `resize_vocab: false` and ensure the tokenizer covers the needed tokens already.","Or drop `quantization_bit` (run BF16) so embedding layers can be resized.","Alternatively resize the model vocabulary offline first, save the modified checkpoint, then quantize and train."],"exampleFix":"# before (yaml)\nquantization_bit: 4\nresize_vocab: true\n\n# after (yaml)\nquantization_bit: 4\nresize_vocab: false","handlingStrategy":"validation","validationCode":"if cfg.get('quantization_bit') is not None and cfg.get('resize_vocab'):\n    raise SystemExit('cannot resize vocab on a quantized model')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pre-resize and re-quantize checkpoints offline when new tokens are needed."],"tags":["quantization","tokenizer","vocabulary","llamafactory"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}