{"record":{"id":"be1c89be6f9433f8","repo":"hiyouga/LlamaFactory","slug":"eetq-quantization-is-incompatible-with-deepspeed-z","errorCode":null,"errorMessage":"EETQ quantization is incompatible with DeepSpeed ZeRO-3 or FSDP.","messagePattern":"EETQ quantization is incompatible with DeepSpeed ZeRO-3 or FSDP\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/model/model_utils/quantization.py","lineNumber":223,"sourceCode":"            logger.info_rank0(f\"Quantizing model to {model_args.quantization_bit} bit with bitsandbytes.\")\n        elif model_args.quantization_method == QuantizationMethod.HQQ:\n            if model_args.quantization_bit not in [8, 6, 5, 4, 3, 2, 1]:\n                raise ValueError(\"HQQ only accepts 1/2/3/4/5/6/8-bit quantization.\")\n\n            if is_deepspeed_zero3_enabled() or is_fsdp_enabled():\n                raise ValueError(\"HQQ quantization is incompatible with DeepSpeed ZeRO-3 or FSDP.\")\n\n            check_version(\"hqq\", mandatory=True)\n            init_kwargs[\"quantization_config\"] = HqqConfig(\n                nbits=model_args.quantization_bit, quant_zero=False, quant_scale=False, axis=0\n            )  # use ATEN kernel (axis=0) for performance\n            logger.info_rank0(f\"Quantizing model to {model_args.quantization_bit} bit with HQQ.\")\n        elif model_args.quantization_method == QuantizationMethod.EETQ:\n            if model_args.quantization_bit != 8:\n                raise ValueError(\"EETQ only accepts 8-bit quantization.\")\n\n            if is_deepspeed_zero3_enabled() or is_fsdp_enabled():\n                raise ValueError(\"EETQ quantization is incompatible with DeepSpeed ZeRO-3 or FSDP.\")\n\n            check_version(\"eetq\", mandatory=True)\n            init_kwargs[\"quantization_config\"] = EetqConfig()\n            logger.info_rank0(f\"Quantizing model to {model_args.quantization_bit} bit with EETQ.\")\n","sourceCodeStart":205,"sourceCodeEnd":228,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/model/model_utils/quantization.py#L205-L228","documentation":"Like HQQ, EETQ-quantized parameters cannot be partitioned by DeepSpeed ZeRO-3 or FSDP. configure_quantization raises ValueError when quantization_method: eetq is combined with either sharding backend, before building EetqConfig.","triggerScenarios":"quantization_method: eetq with quantization_bit: 8 in a run where a ZeRO-3 deepspeed config or FSDP is enabled.","commonSituations":"Multi-GPU EETQ INT8 attempts with a z3 template; migrating a single-GPU EETQ setup to distributed training unchanged.","solutions":["Use bitsandbytes 4-bit (QLoRA) if you need ZeRO-3/FSDP sharding.","Or run EETQ with ZeRO-2 / without FSDP on a single process.","Precompute a PTQ INT8 checkpoint only if your serving stack supports it; training-side sharding stays unsupported."],"exampleFix":"# before (yaml)\nquantization_method: eetq\nquantization_bit: 8\ndeepspeed: examples/deepspeed/ds_z3_config.json\n\n# after (yaml)\nquantization_method: eetq\nquantization_bit: 8\ndeepspeed: examples/deepspeed/ds_z2_config.json","handlingStrategy":"validation","validationCode":"if model_args.quantization_method == \"eetq\":\n    assert not (is_deepspeed_zero3_enabled() or is_fsdp_enabled()), (\n        \"EETQ is incompatible with ZeRO-3/FSDP; use bnb 4-bit or ZeRO-2\"\n    )","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep a compatibility matrix (method x backend) next to training templates.","For sharded multi-GPU runs, restrict quantization methods to bitsandbytes 4-bit."],"tags":["eetq","quantization","deepspeed","fsdp","zero3"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}