{"record":{"id":"777c433d3e7f846e","repo":"hiyouga/LlamaFactory","slug":"eetq-only-accepts-8-bit-quantization","errorCode":null,"errorMessage":"EETQ only accepts 8-bit quantization.","messagePattern":"EETQ only accepts 8-bit quantization\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/model/model_utils/quantization.py","lineNumber":220,"sourceCode":"            else:\n                init_kwargs[\"device_map\"] = {\"\": get_current_device()}  # change auto device map for inference\n\n            logger.info_rank0(f\"Quantizing model to {model_args.quantization_bit} bit with bitsandbytes.\")\n        elif model_args.quantization_method == QuantizationMethod.HQQ:\n            if model_args.quantization_bit not in [8, 6, 5, 4, 3, 2, 1]:\n                raise ValueError(\"HQQ only accepts 1/2/3/4/5/6/8-bit quantization.\")\n\n            if is_deepspeed_zero3_enabled() or is_fsdp_enabled():\n                raise ValueError(\"HQQ quantization is incompatible with DeepSpeed ZeRO-3 or FSDP.\")\n\n            check_version(\"hqq\", mandatory=True)\n            init_kwargs[\"quantization_config\"] = HqqConfig(\n                nbits=model_args.quantization_bit, quant_zero=False, quant_scale=False, axis=0\n            )  # use ATEN kernel (axis=0) for performance\n            logger.info_rank0(f\"Quantizing model to {model_args.quantization_bit} bit with HQQ.\")\n        elif model_args.quantization_method == QuantizationMethod.EETQ:\n            if model_args.quantization_bit != 8:\n                raise ValueError(\"EETQ only accepts 8-bit quantization.\")\n\n            if is_deepspeed_zero3_enabled() or is_fsdp_enabled():\n                raise ValueError(\"EETQ quantization is incompatible with DeepSpeed ZeRO-3 or FSDP.\")\n\n            check_version(\"eetq\", mandatory=True)\n            init_kwargs[\"quantization_config\"] = EetqConfig()\n            logger.info_rank0(f\"Quantizing model to {model_args.quantization_bit} bit with EETQ.\")\n","sourceCodeStart":202,"sourceCodeEnd":228,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/model/model_utils/quantization.py#L202-L228","documentation":"EETQ provides only INT8 weight-only quantization kernels. When quantization_method: eetq is selected, LlamaFactory requires quantization_bit == 8 and raises ValueError otherwise, mirroring the EetqConfig() default that the code path builds.","triggerScenarios":"quantization_method: eetq with quantization_bit set to anything other than 8 (e.g. 4, copying from bnb QLoRA configs).","commonSituations":"Copy-pasting a 4-bit QLoRA template and only changing quantization_method to eetq; assuming method-agnostic bit widths.","solutions":["Set quantization_bit: 8 together with quantization_method: eetq.","If you need 4-bit, use bitsandbytes (train) or export-time GPTQ.","Verify the eetq package is installed (the code will check_version('eetq') next)."],"exampleFix":"# before (yaml)\nquantization_method: eetq\nquantization_bit: 4\n\n# after (yaml)\nquantization_method: eetq\nquantization_bit: 8","handlingStrategy":"validation","validationCode":"if model_args.quantization_method == \"eetq\":\n    assert model_args.quantization_bit == 8, \"EETQ supports only 8-bit\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["EETQ means INT8 only; never copy a 4-bit bnb setting onto it.","Prefer bnb for 4-bit training-side quantization."],"tags":["eetq","quantization","config-validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}