hiyouga/LlamaFactory · error · ValueError

EETQ only accepts 8-bit quantization.

Error message

EETQ only accepts 8-bit quantization.

What it means

EETQ provides only INT8 weight-only quantization kernels. When quantization_method: eetq is selected, LlamaFactory requires quantization_bit == 8 and raises ValueError otherwise, mirroring the EetqConfig() default that the code path builds.

Source

Thrown at src/llamafactory/model/model_utils/quantization.py:220

            else:
                init_kwargs["device_map"] = {"": get_current_device()}  # change auto device map for inference

            logger.info_rank0(f"Quantizing model to {model_args.quantization_bit} bit with bitsandbytes.")
        elif model_args.quantization_method == QuantizationMethod.HQQ:
            if model_args.quantization_bit not in [8, 6, 5, 4, 3, 2, 1]:
                raise ValueError("HQQ only accepts 1/2/3/4/5/6/8-bit quantization.")

            if is_deepspeed_zero3_enabled() or is_fsdp_enabled():
                raise ValueError("HQQ quantization is incompatible with DeepSpeed ZeRO-3 or FSDP.")

            check_version("hqq", mandatory=True)
            init_kwargs["quantization_config"] = HqqConfig(
                nbits=model_args.quantization_bit, quant_zero=False, quant_scale=False, axis=0
            )  # use ATEN kernel (axis=0) for performance
            logger.info_rank0(f"Quantizing model to {model_args.quantization_bit} bit with HQQ.")
        elif model_args.quantization_method == QuantizationMethod.EETQ:
            if model_args.quantization_bit != 8:
                raise ValueError("EETQ only accepts 8-bit quantization.")

            if is_deepspeed_zero3_enabled() or is_fsdp_enabled():
                raise ValueError("EETQ quantization is incompatible with DeepSpeed ZeRO-3 or FSDP.")

            check_version("eetq", mandatory=True)
            init_kwargs["quantization_config"] = EetqConfig()
            logger.info_rank0(f"Quantizing model to {model_args.quantization_bit} bit with EETQ.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Set quantization_bit: 8 together with quantization_method: eetq.
2. If you need 4-bit, use bitsandbytes (train) or export-time GPTQ.
3. Verify the eetq package is installed (the code will check_version('eetq') next).

Example fix

# before (yaml)
quantization_method: eetq
quantization_bit: 4

# after (yaml)
quantization_method: eetq
quantization_bit: 8

if model_args.quantization_method == "eetq":
    assert model_args.quantization_bit == 8, "EETQ supports only 8-bit"

When it happens

Trigger: quantization_method: eetq with quantization_bit set to anything other than 8 (e.g. 4, copying from bnb QLoRA configs).

Common situations: Copy-pasting a 4-bit QLoRA template and only changing quantization_method to eetq; assuming method-agnostic bit widths.

Related errors

• Cannot use device map for quantized models in training.
• FP8 training is not compatible with quantization. Please dis
• AutoGPTQ only accepts 2/3/4/8-bit quantization.
• Bitsandbytes only accepts 4-bit or 8-bit quantization.
• HQQ only accepts 1/2/3/4/5/6/8-bit quantization.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/777c433d3e7f846e. Report an issue: GitHub.