hiyouga/LlamaFactory · error · ValueError

`predict_with_generate` cannot be set as True except SFT.

Error message

`predict_with_generate` cannot be set as True except SFT.

What it means

predict_with_generate makes the trainer run generation during evaluation to produce text predictions, which is only implemented for the SFT stage (it needs an instruction-tuned decode path with the model's generate()). For stages like pt/rm/ppo/dpo/kto the flag is meaningless, so the parser rejects it when finetuning_args.stage != 'sft'.

Source

Thrown at src/llamafactory/hparams/parser.py:420

                "megatron-bridge is required when USE_MEGATRON_BRIDGE=1. "
                "Please install `megatron-bridge` and its dependencies."
            )
        model_args, data_args, training_args, finetuning_args, mb_args, generating_args = _parse_train_mbridge_args(
            args
        )
    else:
        model_args, data_args, training_args, finetuning_args, generating_args = _parse_train_args(args)
        finetuning_args.use_mca = False
        finetuning_args.use_megatron_bridge = False

    # Setup logging
    if training_args.should_log:
        _set_transformers_logging()

    # Check arguments
    if finetuning_args.stage != "sft":
        if training_args.predict_with_generate:
            raise ValueError("`predict_with_generate` cannot be set as True except SFT.")

        if data_args.neat_packing:
            raise ValueError("`neat_packing` cannot be set as True except SFT.")

        if data_args.train_on_prompt or data_args.mask_history:
            raise ValueError("`train_on_prompt` or `mask_history` cannot be set as True except SFT.")

    if finetuning_args.stage == "sft" and training_args.do_predict and not training_args.predict_with_generate:
        raise ValueError("Please enable `predict_with_generate` to save model predictions.")

    if finetuning_args.use_megatron_bridge:
        if finetuning_args.use_mca or finetuning_args.use_hyper_parallel:
            raise ValueError("Megatron Bridge cannot be used together with MCA or HyperParallel.")
        if finetuning_args.stage not in ["pt", "sft"]:
            raise ValueError("Megatron Bridge only supports the `pt` and `sft` stages.")
        if finetuning_args.finetuning_type not in ["full", "lora"]:
            raise ValueError("Megatron Bridge only supports `full` and `lora` finetuning.")
        if model_args.quantization_bit is not None:

View on GitHub (pinned to f28afaf635)

Solutions

1. Set predict_with_generate: false in the training_args block of your config.
2. If you actually want generation-based evaluation, change stage: sft and use the SFT pipeline.
3. Remove do_predict/do_eval-with-generation from non-SFT configs entirely.

Example fix

# before
stage: dpo
predict_with_generate: true

# after
stage: dpo
predict_with_generate: false

stage = cfg.get("stage", "sft")
if stage != "sft" and cfg.get("predict_with_generate"):
    raise SystemExit(f"predict_with_generate is SFT-only; config stage is {stage}")

When it happens

Trigger: A train config with stage: rm (or dpo/ppo/pt/kto) together with predict_with_generate: true, passed to llamafactory-cli train / run_exp(). Typical when copying an SFT eval config and only changing the stage field.

Common situations: Reusing a YAML that was written for SFT with do_eval + predict_with_generate, then switching stage to dpo or rm for preference training; webui-generated configs that keep the prediction flag enabled.

Related errors

• `neat_packing` cannot be set as True except SFT.
• `train_on_prompt` or `mask_history` cannot be set as True ex
• Please enable `predict_with_generate` to save model predicti
• Megatron Bridge only supports the `pt` and `sft` stages.
• PPO training does not support evaluation, use the SFT stage

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/ee20eee068922730. Report an issue: GitHub.