hiyouga/LlamaFactory · error · ValueError

Cannot resize embedding layers of a quantized model.

Error message

Cannot resize embedding layers of a quantized model.

What it means

Raised by _verify_model_args when quantization_bit is set and resize_vocab is true. Resizing embedding layers rewrites the token-embedding matrix with new randomly initialized rows; on a quantized model those weights are stored in compressed INT4/INT8 form and cannot be resized in place.

Source

Thrown at src/llamafactory/hparams/parser.py:243

def _verify_model_args(
    model_args: "ModelArguments",
    data_args: "DataArguments",
    finetuning_args: "FinetuningArguments",
) -> None:
    if model_args.adapter_name_or_path is not None and finetuning_args.finetuning_type != "lora":
        raise ValueError("Adapter is only valid for the LoRA method.")

    if model_args.quantization_bit is not None:
        if finetuning_args.finetuning_type not in ["lora", "oft"]:
            raise ValueError("Quantization is only compatible with the LoRA or OFT method.")

        if finetuning_args.pissa_init:
            raise ValueError("Please use scripts/pissa_init.py to initialize PiSSA for a quantized model.")

        if model_args.resize_vocab:
            raise ValueError("Cannot resize embedding layers of a quantized model.")

        if model_args.adapter_name_or_path is not None and finetuning_args.create_new_adapter:
            raise ValueError("Cannot create new adapter upon a quantized model.")

        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:
            raise ValueError("Quantized model only accepts a single adapter. Merge them first.")


def _check_extra_dependencies(
    model_args: "ModelArguments",
    finetuning_args: "FinetuningArguments",
    training_args: Optional["TrainingArguments"] = None,
) -> None:
    if model_args.use_kt:
        check_version("kt-kernel", mandatory=True)
        check_version("transformers-kt", mandatory=True)
        check_version("accelerate-kt", mandatory=True)

View on GitHub (pinned to f28afaf635)

Solutions

1. Set `resize_vocab: false` and ensure the tokenizer covers the needed tokens already.
2. Or drop `quantization_bit` (run BF16) so embedding layers can be resized.
3. Alternatively resize the model vocabulary offline first, save the modified checkpoint, then quantize and train.

Example fix

# before (yaml)
quantization_bit: 4
resize_vocab: true

# after (yaml)
quantization_bit: 4
resize_vocab: false

if cfg.get('quantization_bit') is not None and cfg.get('resize_vocab'):
    raise SystemExit('cannot resize vocab on a quantized model')

When it happens

Trigger: A YAML combining quantization_bit: 4/8 with resize_vocab: true (typically added to support added special tokens after training on a bigger tokenizer); _verify_model_args rejects it before loading.

Common situations: Users fine-tune with additional tokens (chat templates, tool markers) and keep QLoRA quantization on, expecting vocab resizing to work as in BF16 runs.

Related errors

• Quantization is only compatible with the LoRA or OFT method.
• Please use scripts/pissa_init.py to initialize PiSSA for a q
• Cannot resize embedding layers of a quantized model.
• Stop words are required to replace the EOS token.
• YAML config must be a dictionary mapping tokens to descripti

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/46a71965356f2143. Report an issue: GitHub.