{"record":{"id":"795aa4a07d398d3e","repo":"hiyouga/LlamaFactory","slug":"train-on-prompt-or-mask-history-cannot-be-set","errorCode":null,"errorMessage":"`train_on_prompt` or `mask_history` cannot be set as True except SFT.","messagePattern":"`train_on_prompt` or `mask_history` cannot be set as True except SFT\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":426,"sourceCode":"    else:\n        model_args, data_args, training_args, finetuning_args, generating_args = _parse_train_args(args)\n        finetuning_args.use_mca = False\n        finetuning_args.use_megatron_bridge = False\n\n    # Setup logging\n    if training_args.should_log:\n        _set_transformers_logging()\n\n    # Check arguments\n    if finetuning_args.stage != \"sft\":\n        if training_args.predict_with_generate:\n            raise ValueError(\"`predict_with_generate` cannot be set as True except SFT.\")\n\n        if data_args.neat_packing:\n            raise ValueError(\"`neat_packing` cannot be set as True except SFT.\")\n\n        if data_args.train_on_prompt or data_args.mask_history:\n            raise ValueError(\"`train_on_prompt` or `mask_history` cannot be set as True except SFT.\")\n\n    if finetuning_args.stage == \"sft\" and training_args.do_predict and not training_args.predict_with_generate:\n        raise ValueError(\"Please enable `predict_with_generate` to save model predictions.\")\n\n    if finetuning_args.use_megatron_bridge:\n        if finetuning_args.use_mca or finetuning_args.use_hyper_parallel:\n            raise ValueError(\"Megatron Bridge cannot be used together with MCA or HyperParallel.\")\n        if finetuning_args.stage not in [\"pt\", \"sft\"]:\n            raise ValueError(\"Megatron Bridge only supports the `pt` and `sft` stages.\")\n        if finetuning_args.finetuning_type not in [\"full\", \"lora\"]:\n            raise ValueError(\"Megatron Bridge only supports `full` and `lora` finetuning.\")\n        if model_args.quantization_bit is not None:\n            raise ValueError(\"Quantized models are not supported with Megatron Bridge.\")\n        if training_args.deepspeed is not None:\n            raise ValueError(\"Megatron Bridge is incompatible with DeepSpeed.\")\n        if mb_args is None:\n            raise ValueError(\"Megatron Bridge arguments are missing. Please set USE_MEGATRON_BRIDGE=1.\")\n        _validate_megatron_bridge_parallel_args(mb_args, training_args.world_size)","sourceCodeStart":408,"sourceCodeEnd":444,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L408-L444","documentation":"train_on_prompt (train on prompt tokens too) and mask_history (mask earlier conversation turns) are labels-masking options implemented only in the SFT supervised processor. For pt/dpo/rm/ppo/kto stages the label construction differs (e.g. preference pairs, scalar rewards), so either flag with a non-SFT stage is rejected at parse time.","triggerScenarios":"A train config with stage != sft plus train_on_prompt: true or mask_history: true under data_args. Raised by get_train_args() before data processing begins.","commonSituations":"Porting a chat-SFT YAML (which often sets mask_history: true) to a DPO or RM stage by only editing stage and dataset; enabling train_on_prompt for continued pretraining (pt) where it has no meaning.","solutions":["Set train_on_prompt: false and mask_history: false (or delete those keys) for non-SFT stages.","Keep such masking options only in stage: sft configs.","Template-level control (e.g. empty-system or template choice) is the right place to adjust prompt content for other stages."],"exampleFix":"# before\nstage: dpo\ntrain_on_prompt: true\nmask_history: true\n\n# after\nstage: dpo\ntrain_on_prompt: false\nmask_history: false","handlingStrategy":"validation","validationCode":"if cfg.get(\"stage\", \"sft\") != \"sft\" and (cfg.get(\"train_on_prompt\") or cfg.get(\"mask_history\")):\n    raise SystemExit(\"train_on_prompt/mask_history are SFT-only flags\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep a whitelist of per-stage keys and validate configs against it before submission.","When converting an SFT chat config to DPO/RM, delete the data-mask block rather than toggling values."],"tags":["config","validation","data-mask","stage","sft"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}