{"record":{"id":"02db58655459c0ab","repo":"hiyouga/LlamaFactory","slug":"neat-packing-cannot-be-set-as-true-except-sft","errorCode":null,"errorMessage":"`neat_packing` cannot be set as True except SFT.","messagePattern":"`neat_packing` cannot be set as True except SFT\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":423,"sourceCode":"        model_args, data_args, training_args, finetuning_args, mb_args, generating_args = _parse_train_mbridge_args(\n            args\n        )\n    else:\n        model_args, data_args, training_args, finetuning_args, generating_args = _parse_train_args(args)\n        finetuning_args.use_mca = False\n        finetuning_args.use_megatron_bridge = False\n\n    # Setup logging\n    if training_args.should_log:\n        _set_transformers_logging()\n\n    # Check arguments\n    if finetuning_args.stage != \"sft\":\n        if training_args.predict_with_generate:\n            raise ValueError(\"`predict_with_generate` cannot be set as True except SFT.\")\n\n        if data_args.neat_packing:\n            raise ValueError(\"`neat_packing` cannot be set as True except SFT.\")\n\n        if data_args.train_on_prompt or data_args.mask_history:\n            raise ValueError(\"`train_on_prompt` or `mask_history` cannot be set as True except SFT.\")\n\n    if finetuning_args.stage == \"sft\" and training_args.do_predict and not training_args.predict_with_generate:\n        raise ValueError(\"Please enable `predict_with_generate` to save model predictions.\")\n\n    if finetuning_args.use_megatron_bridge:\n        if finetuning_args.use_mca or finetuning_args.use_hyper_parallel:\n            raise ValueError(\"Megatron Bridge cannot be used together with MCA or HyperParallel.\")\n        if finetuning_args.stage not in [\"pt\", \"sft\"]:\n            raise ValueError(\"Megatron Bridge only supports the `pt` and `sft` stages.\")\n        if finetuning_args.finetuning_type not in [\"full\", \"lora\"]:\n            raise ValueError(\"Megatron Bridge only supports `full` and `lora` finetuning.\")\n        if model_args.quantization_bit is not None:\n            raise ValueError(\"Quantized models are not supported with Megatron Bridge.\")\n        if training_args.deepspeed is not None:\n            raise ValueError(\"Megatron Bridge is incompatible with DeepSpeed.\")","sourceCodeStart":405,"sourceCodeEnd":441,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L405-L441","documentation":"neat_packing is a sequence-packing strategy (with position-id based attention separation) implemented only in the SFT supervised data processor. When stage is anything other than sft, the flag has no code path and the argument checker raises ValueError to prevent silently ignored packing.","triggerScenarios":"A config with stage != sft (e.g. dpo, rm, ppo, pt, kto) and neat_packing: true in data_args, submitted via llamafactory-cli train. Also triggered programmatically through run_exp(dict) with the same combination.","commonSituations":"Copy-pasting an efficient SFT config (packing for throughput) into a DPO/RM run; upgrading from an older version where the flag was silently ignored and now validates strictly.","solutions":["Remove neat_packing or set it to false for non-SFT stages.","If packing matters for throughput, ensure the run is stage: sft.","For long-sequence efficiency in DPO/RM, use cutoff_len and batching appropriate to that stage instead."],"exampleFix":"# before\nstage: dpo\nneat_packing: true\n\n# after\nstage: dpo\nneat_packing: false","handlingStrategy":"validation","validationCode":"if cfg.get(\"stage\", \"sft\") != \"sft\" and cfg.get(\"neat_packing\"):\n    raise SystemExit(\"neat_packing is SFT-only; remove it for this stage\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat packing flags as SFT-template fields; strip them in any script that repurposes SFT YAMLs for other stages.","Automate config diffs when switching stages so leftover stage-specific flags are surfaced."],"tags":["config","validation","packing","stage","sft"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}