hiyouga/LlamaFactory · error · ValueError

FP8 training is not compatible with quantization. Please dis

Error message

FP8 training is not compatible with quantization. Please disable one of them.

What it means

Raised in parser.py:527 when training_args.fp8 is true and model_args.quantization_bit is not None (unless use_mca or use_megatron_bridge is on). FP8 training and weight quantization (GPTQ/AWQ/bitsandbytes) are two separate precision reductions; applying both simultaneously is unsupported and would compound accuracy loss.

Source

Thrown at src/llamafactory/hparams/parser.py:527

        if finetuning_args.use_apollo and finetuning_args.apollo_layerwise:
            raise ValueError("Distributed training does not support layer-wise APOLLO.")

        if finetuning_args.use_badam:
            if finetuning_args.badam_mode == "ratio":
                raise ValueError("Radio-based BAdam does not yet support distributed training, use layer-wise BAdam.")
            elif not is_deepspeed_zero3_enabled():
                raise ValueError("Layer-wise BAdam only supports DeepSpeed ZeRO-3 training.")

    if training_args.deepspeed is not None and (finetuning_args.use_galore or finetuning_args.use_apollo):
        raise ValueError("GaLore and APOLLO are incompatible with DeepSpeed yet.")

    if (
        not finetuning_args.use_mca
        and not finetuning_args.use_megatron_bridge
        and training_args.fp8
        and model_args.quantization_bit is not None
    ):
        raise ValueError("FP8 training is not compatible with quantization. Please disable one of them.")

    if model_args.infer_backend != EngineName.HF:
        raise ValueError("vLLM/SGLang backend is only available for API, CLI and Web.")

    if model_args.use_unsloth and is_deepspeed_zero3_enabled():
        raise ValueError("Unsloth is incompatible with DeepSpeed ZeRO-3.")

    if model_args.use_kt and is_deepspeed_zero3_enabled():
        raise ValueError("KTransformers is incompatible with DeepSpeed ZeRO-3.")

    _set_env_vars()
    _verify_model_args(model_args, data_args, finetuning_args)
    _check_extra_dependencies(model_args, finetuning_args, training_args)
    _verify_trackio_args(training_args)

    if (
        not finetuning_args.use_mca
        and not finetuning_args.use_megatron_bridge

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove `quantization_bit` (and quantization_method) to do pure FP8 training
2. Or disable `fp8: true` to do standard quantized (Q)LoRA training
3. Verify you are on FP8-capable hardware (H100/H200/Ada) before keeping fp8

Example fix

# before (YAML)
fp8: true
quantization_bit: 4

# after
fp8: true
# quantization_bit removed

if config.get("fp8") and config.get("quantization_bit") is not None:
    raise SystemExit("Choose FP8 training OR quantization, not both")

When it happens

Trigger: Config containing `fp8: true` together with `quantization_bit: 8` (or 4/2) and any quantization_method, outside MCA/Megatron-bridge runs.

Common situations: Enabling FP8 (Transformer Engine) on H100 while forgetting a leftover quantization_bit from an earlier QLoRA experiment; merging a QLoRA config with an FP8 template.

Related errors

• Cannot use device map for quantized models in training.
• This device does not support `pure_bf16`.
• `quantization_bit` cannot be combined with KT weight caches.
• AutoGPTQ only accepts 2/3/4/8-bit quantization.
• Bitsandbytes only accepts 4-bit or 8-bit quantization.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/5dcfde357f7e67c9. Report an issue: GitHub.