{"record":{"id":"5dcfde357f7e67c9","repo":"hiyouga/LlamaFactory","slug":"fp8-training-is-not-compatible-with-quantization","errorCode":null,"errorMessage":"FP8 training is not compatible with quantization. Please disable one of them.","messagePattern":"FP8 training is not compatible with quantization\\. Please disable one of them\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":527,"sourceCode":"        if finetuning_args.use_apollo and finetuning_args.apollo_layerwise:\n            raise ValueError(\"Distributed training does not support layer-wise APOLLO.\")\n\n        if finetuning_args.use_badam:\n            if finetuning_args.badam_mode == \"ratio\":\n                raise ValueError(\"Radio-based BAdam does not yet support distributed training, use layer-wise BAdam.\")\n            elif not is_deepspeed_zero3_enabled():\n                raise ValueError(\"Layer-wise BAdam only supports DeepSpeed ZeRO-3 training.\")\n\n    if training_args.deepspeed is not None and (finetuning_args.use_galore or finetuning_args.use_apollo):\n        raise ValueError(\"GaLore and APOLLO are incompatible with DeepSpeed yet.\")\n\n    if (\n        not finetuning_args.use_mca\n        and not finetuning_args.use_megatron_bridge\n        and training_args.fp8\n        and model_args.quantization_bit is not None\n    ):\n        raise ValueError(\"FP8 training is not compatible with quantization. Please disable one of them.\")\n\n    if model_args.infer_backend != EngineName.HF:\n        raise ValueError(\"vLLM/SGLang backend is only available for API, CLI and Web.\")\n\n    if model_args.use_unsloth and is_deepspeed_zero3_enabled():\n        raise ValueError(\"Unsloth is incompatible with DeepSpeed ZeRO-3.\")\n\n    if model_args.use_kt and is_deepspeed_zero3_enabled():\n        raise ValueError(\"KTransformers is incompatible with DeepSpeed ZeRO-3.\")\n\n    _set_env_vars()\n    _verify_model_args(model_args, data_args, finetuning_args)\n    _check_extra_dependencies(model_args, finetuning_args, training_args)\n    _verify_trackio_args(training_args)\n\n    if (\n        not finetuning_args.use_mca\n        and not finetuning_args.use_megatron_bridge","sourceCodeStart":509,"sourceCodeEnd":545,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L509-L545","documentation":"Raised in parser.py:527 when training_args.fp8 is true and model_args.quantization_bit is not None (unless use_mca or use_megatron_bridge is on). FP8 training and weight quantization (GPTQ/AWQ/bitsandbytes) are two separate precision reductions; applying both simultaneously is unsupported and would compound accuracy loss.","triggerScenarios":"Config containing `fp8: true` together with `quantization_bit: 8` (or 4/2) and any quantization_method, outside MCA/Megatron-bridge runs.","commonSituations":"Enabling FP8 (Transformer Engine) on H100 while forgetting a leftover quantization_bit from an earlier QLoRA experiment; merging a QLoRA config with an FP8 template.","solutions":["Remove `quantization_bit` (and quantization_method) to do pure FP8 training","Or disable `fp8: true` to do standard quantized (Q)LoRA training","Verify you are on FP8-capable hardware (H100/H200/Ada) before keeping fp8"],"exampleFix":"# before (YAML)\nfp8: true\nquantization_bit: 4\n\n# after\nfp8: true\n# quantization_bit removed","handlingStrategy":"validation","validationCode":"if config.get(\"fp8\") and config.get(\"quantization_bit\") is not None:\n    raise SystemExit(\"Choose FP8 training OR quantization, not both\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Audit leftover quantization keys before enabling fp8","Confirm FP8-capable hardware (H100/Ada+) before planning fp8 runs"],"tags":["fp8","quantization","precision","incompatible-flags","config-validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}