hiyouga/LlamaFactory · error · ValueError

Quantized models are not supported with Megatron Bridge.

Error message

Quantized models are not supported with Megatron Bridge.

What it means

Megatron Bridge loads and shards full-precision weights into its own distributed format; quantized (bitsandbytes/GPTQ-style) checkpoints are incompatible with that path. The parser rejects any config where model_args.quantization_bit is not None while use_megatron_bridge is true.

Source

Thrown at src/llamafactory/hparams/parser.py:439

        if data_args.neat_packing:
            raise ValueError("`neat_packing` cannot be set as True except SFT.")

        if data_args.train_on_prompt or data_args.mask_history:
            raise ValueError("`train_on_prompt` or `mask_history` cannot be set as True except SFT.")

    if finetuning_args.stage == "sft" and training_args.do_predict and not training_args.predict_with_generate:
        raise ValueError("Please enable `predict_with_generate` to save model predictions.")

    if finetuning_args.use_megatron_bridge:
        if finetuning_args.use_mca or finetuning_args.use_hyper_parallel:
            raise ValueError("Megatron Bridge cannot be used together with MCA or HyperParallel.")
        if finetuning_args.stage not in ["pt", "sft"]:
            raise ValueError("Megatron Bridge only supports the `pt` and `sft` stages.")
        if finetuning_args.finetuning_type not in ["full", "lora"]:
            raise ValueError("Megatron Bridge only supports `full` and `lora` finetuning.")
        if model_args.quantization_bit is not None:
            raise ValueError("Quantized models are not supported with Megatron Bridge.")
        if training_args.deepspeed is not None:
            raise ValueError("Megatron Bridge is incompatible with DeepSpeed.")
        if mb_args is None:
            raise ValueError("Megatron Bridge arguments are missing. Please set USE_MEGATRON_BRIDGE=1.")
        _validate_megatron_bridge_parallel_args(mb_args, training_args.world_size)
        finetuning_args.megatron_bridge_args = mb_args

    if finetuning_args.stage in ["rm", "ppo"] and training_args.load_best_model_at_end:
        raise ValueError("RM and PPO stages do not support `load_best_model_at_end`.")

    if finetuning_args.stage == "ppo":
        if not training_args.do_train:
            raise ValueError("PPO training does not support evaluation, use the SFT stage to evaluate models.")

        if model_args.shift_attn:
            raise ValueError("PPO training is incompatible with S^2-Attn.")

        if finetuning_args.reward_model_type == "lora" and model_args.use_kt:

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove quantization_bit (or set it to null) for Megatron Bridge runs; rely on TP/PP sharding for memory.
2. If you must train quantized, drop Megatron Bridge (unset USE_MEGATRON_BRIDGE) and use the standard QLoRA path.
3. For deployment-size reduction, quantize after training during export instead.

Example fix

# before
export USE_MEGATRON_BRIDGE=1
quantization_bit: 4

# after
export USE_MEGATRON_BRIDGE=1
quantization_bit:  # removed / null

import os

if os.environ.get("USE_MEGATRON_BRIDGE") == "1" and cfg.get("quantization_bit") is not None:
    raise SystemExit("Remove quantization_bit for Megatron Bridge runs")

When it happens

Trigger: USE_MEGATRON_BRIDGE=1 plus quantization_bit: 4 (or 8) in the model_args block of the training config.

Common situations: Memory-saving QLoRA configs reused when switching to Megatron Bridge for multi-GPU throughput; users assuming quantization carries over between backends.

Related errors

• Total Megatron Bridge parallel size ({parallel_size}) exceed
• Total Megatron Bridge parallel size ({parallel_size}) must d
• Megatron Bridge cannot be used together with MCA or HyperPar
• Megatron Bridge only supports the `pt` and `sft` stages.
• Megatron Bridge only supports `full` and `lora` finetuning.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/2b616857a01072ab. Report an issue: GitHub.