hiyouga/LlamaFactory · error · ValueError

Please enable `predict_with_generate` to save model predicti

Error message

Please enable `predict_with_generate` to save model predictions.

What it means

In SFT stage, running prediction (do_predict: true) without predict_with_generate would only emit logits/tensors, which is useless for text model predictions. The parser forces you to opt into generation so that predictions.json contains actual generated text via the model's generate() path.

Source

Thrown at src/llamafactory/hparams/parser.py:429

        finetuning_args.use_megatron_bridge = False

    # Setup logging
    if training_args.should_log:
        _set_transformers_logging()

    # Check arguments
    if finetuning_args.stage != "sft":
        if training_args.predict_with_generate:
            raise ValueError("`predict_with_generate` cannot be set as True except SFT.")

        if data_args.neat_packing:
            raise ValueError("`neat_packing` cannot be set as True except SFT.")

        if data_args.train_on_prompt or data_args.mask_history:
            raise ValueError("`train_on_prompt` or `mask_history` cannot be set as True except SFT.")

    if finetuning_args.stage == "sft" and training_args.do_predict and not training_args.predict_with_generate:
        raise ValueError("Please enable `predict_with_generate` to save model predictions.")

    if finetuning_args.use_megatron_bridge:
        if finetuning_args.use_mca or finetuning_args.use_hyper_parallel:
            raise ValueError("Megatron Bridge cannot be used together with MCA or HyperParallel.")
        if finetuning_args.stage not in ["pt", "sft"]:
            raise ValueError("Megatron Bridge only supports the `pt` and `sft` stages.")
        if finetuning_args.finetuning_type not in ["full", "lora"]:
            raise ValueError("Megatron Bridge only supports `full` and `lora` finetuning.")
        if model_args.quantization_bit is not None:
            raise ValueError("Quantized models are not supported with Megatron Bridge.")
        if training_args.deepspeed is not None:
            raise ValueError("Megatron Bridge is incompatible with DeepSpeed.")
        if mb_args is None:
            raise ValueError("Megatron Bridge arguments are missing. Please set USE_MEGATRON_BRIDGE=1.")
        _validate_megatron_bridge_parallel_args(mb_args, training_args.world_size)
        finetuning_args.megatron_bridge_args = mb_args

    if finetuning_args.stage in ["rm", "ppo"] and training_args.load_best_model_at_end:

View on GitHub (pinned to f28afaf635)

Solutions

1. Add predict_with_generate: true to the training_args of your SFT config.
2. If you only want loss/metrics on a held-out set, use do_eval with an eval_dataset instead of do_predict.
3. Optionally set generation config (e.g. temperature, max_new_tokens via generating_args) at the same time to control the outputs.

Example fix

# before
stage: sft
do_predict: true
# predict_with_generate missing

# after
stage: sft
do_predict: true
predict_with_generate: true

if cfg.get("stage", "sft") == "sft" and cfg.get("do_predict") and not cfg.get("predict_with_generate"):
    raise SystemExit("SFT do_predict requires predict_with_generate: true")

When it happens

Trigger: stage: sft together with do_predict: true and predict_with_generate: false (or absent) in training_args, executed with llamafactory-cli train. Common when adding a predict dataset to an existing SFT config.

Common situations: Users adding eval_dataset/predict dataset for qualitative outputs; running llamafactory-cli train x.yaml then llamafactory-cli train x.yaml --do_predict without also flipping predict_with_generate.

Related errors

• `predict_with_generate` cannot be set as True except SFT.
• `neat_packing` cannot be set as True except SFT.
• `train_on_prompt` or `mask_history` cannot be set as True ex
• Unknown mixing strategy: {data_args.mix_strategy}.
• Cannot specify `val_size` if `eval_dataset` is not None.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/937a3cb41c8e410b. Report an issue: GitHub.