{"record":{"id":"937a3cb41c8e410b","repo":"hiyouga/LlamaFactory","slug":"please-enable-predict-with-generate-to-save-mode","errorCode":null,"errorMessage":"Please enable `predict_with_generate` to save model predictions.","messagePattern":"Please enable `predict_with_generate` to save model predictions\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":429,"sourceCode":"        finetuning_args.use_megatron_bridge = False\n\n    # Setup logging\n    if training_args.should_log:\n        _set_transformers_logging()\n\n    # Check arguments\n    if finetuning_args.stage != \"sft\":\n        if training_args.predict_with_generate:\n            raise ValueError(\"`predict_with_generate` cannot be set as True except SFT.\")\n\n        if data_args.neat_packing:\n            raise ValueError(\"`neat_packing` cannot be set as True except SFT.\")\n\n        if data_args.train_on_prompt or data_args.mask_history:\n            raise ValueError(\"`train_on_prompt` or `mask_history` cannot be set as True except SFT.\")\n\n    if finetuning_args.stage == \"sft\" and training_args.do_predict and not training_args.predict_with_generate:\n        raise ValueError(\"Please enable `predict_with_generate` to save model predictions.\")\n\n    if finetuning_args.use_megatron_bridge:\n        if finetuning_args.use_mca or finetuning_args.use_hyper_parallel:\n            raise ValueError(\"Megatron Bridge cannot be used together with MCA or HyperParallel.\")\n        if finetuning_args.stage not in [\"pt\", \"sft\"]:\n            raise ValueError(\"Megatron Bridge only supports the `pt` and `sft` stages.\")\n        if finetuning_args.finetuning_type not in [\"full\", \"lora\"]:\n            raise ValueError(\"Megatron Bridge only supports `full` and `lora` finetuning.\")\n        if model_args.quantization_bit is not None:\n            raise ValueError(\"Quantized models are not supported with Megatron Bridge.\")\n        if training_args.deepspeed is not None:\n            raise ValueError(\"Megatron Bridge is incompatible with DeepSpeed.\")\n        if mb_args is None:\n            raise ValueError(\"Megatron Bridge arguments are missing. Please set USE_MEGATRON_BRIDGE=1.\")\n        _validate_megatron_bridge_parallel_args(mb_args, training_args.world_size)\n        finetuning_args.megatron_bridge_args = mb_args\n\n    if finetuning_args.stage in [\"rm\", \"ppo\"] and training_args.load_best_model_at_end:","sourceCodeStart":411,"sourceCodeEnd":447,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L411-L447","documentation":"In SFT stage, running prediction (do_predict: true) without predict_with_generate would only emit logits/tensors, which is useless for text model predictions. The parser forces you to opt into generation so that predictions.json contains actual generated text via the model's generate() path.","triggerScenarios":"stage: sft together with do_predict: true and predict_with_generate: false (or absent) in training_args, executed with llamafactory-cli train. Common when adding a predict dataset to an existing SFT config.","commonSituations":"Users adding eval_dataset/predict dataset for qualitative outputs; running llamafactory-cli train x.yaml then llamafactory-cli train x.yaml --do_predict without also flipping predict_with_generate.","solutions":["Add predict_with_generate: true to the training_args of your SFT config.","If you only want loss/metrics on a held-out set, use do_eval with an eval_dataset instead of do_predict.","Optionally set generation config (e.g. temperature, max_new_tokens via generating_args) at the same time to control the outputs."],"exampleFix":"# before\nstage: sft\ndo_predict: true\n# predict_with_generate missing\n\n# after\nstage: sft\ndo_predict: true\npredict_with_generate: true","handlingStrategy":"validation","validationCode":"if cfg.get(\"stage\", \"sft\") == \"sft\" and cfg.get(\"do_predict\") and not cfg.get(\"predict_with_generate\"):\n    raise SystemExit(\"SFT do_predict requires predict_with_generate: true\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Whenever you add do_predict to an SFT config, add predict_with_generate: true in the same edit.","Prefer do_eval for metric-only evaluation; reserve do_predict for when you actually want generated text."],"tags":["config","validation","prediction","sft","generation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}