{"record":{"id":"e6e76cb04647c64b","repo":"hiyouga/LlamaFactory","slug":"padding-free-requires-flash-attn-flash-attenti","errorCode":null,"errorMessage":"`padding_free` requires `flash_attn: flash_attention_2`.","messagePattern":"`padding_free` requires `flash_attn: flash_attention_2`\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/core/base_trainer.py","lineNumber":169,"sourceCode":"            if model.config.model_type == \"qwen3_5\":\n                raise RuntimeError(\n                    \"Sequence parallel is not supported for qwen3.5 model due to its different attention implementation, which will be supported in the future.\"\n                )\n            from ..plugins.model_plugins.parallelization.sequence_parallel import SequenceParallelModelPlugin\n\n            if model.config._attn_implementation != \"flash_attention_2\":\n                raise ValueError(\n                    \"Sequence parallelism requires flash attention. Please set `flash_attn: flash_attention_2`.\"\n                )\n\n            SequenceParallelModelPlugin(self.args.cp_mode)(model, self.args.cp_size)\n\n    def _create_batch_generator(self) -> None:\n        if (\n            self.args.batching_strategy == BatchingStrategy.PADDING_FREE\n            and getattr(self.model.config, \"_attn_implementation\", None) != \"flash_attention_2\"\n        ):\n            raise ValueError(\"`padding_free` requires `flash_attn: flash_attention_2`.\")\n\n        self.train_batch_generator = BatchGenerator(\n            dataset=self.train_dataset,\n            renderer=self.renderer,\n            micro_batch_size=self.args.micro_batch_size,\n            global_batch_size=self.args.global_batch_size,\n            cutoff_len=self.args.cutoff_len,\n            batching_workers=self.args.batching_workers,\n            batching_strategy=self.args.batching_strategy,\n            seed=self.args.seed,\n        )\n\n    def _shard_model(self) -> None:\n        if self.args.dist_config is None:\n            if DistributedInterface().get_world_size(Dim.DP) > 1:\n                from torch.nn.parallel import DistributedDataParallel as DDP\n\n                logger.warning_rank0(","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/core/base_trainer.py#L151-L187","documentation":"v1 `BaseTrainer._create_batch_generator` rejects `batching_strategy: padding_free` unless the model's attention implementation is `flash_attention_2`. Padding-free batching packs sequences of different lengths into one varlen batch, which only flash-attention's varlen kernel can score correctly; padded SDPA/eager batches would mix tokens across sequences.","triggerScenarios":"Setting `batching_strategy: padding_free` while the model runs with sdpa/eager attention (flash-attn missing, `flash_attn: auto` fell back, or explicitly set to a non-FA2 value).","commonSituations":"CPU or older-GPU environments where flash-attn is not installed; enabling packing/padding-free to speed up training without checking kernel availability.","solutions":["Set `flash_attn: flash_attention_2` in model args and install flash-attn for your torch/CUDA combo","If flash-attn cannot be installed, use a padding or packing strategy that does not require FA2","Verify the resolved implementation after model load (log `model.config._attn_implementation`) if `auto` is used"],"exampleFix":"# before (yaml)\nmodel:\n  flash_attn: sdpa\ntraining:\n  batching_strategy: padding_free\n\n# after (yaml)\nmodel:\n  flash_attn: flash_attention_2\ntraining:\n  batching_strategy: padding_free","handlingStrategy":"validation","validationCode":"def validate_padding_free(flash_attn: str, strategy: str) -> None:\n    if strategy == \"padding_free\" and flash_attn != \"flash_attention_2\":\n        raise SystemExit(\"padding_free requires flash_attn: flash_attention_2\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat padding_free as flash-attn-only in config templates","Verify flash-attn importability before choosing a batching strategy","Confirm resolved `_attn_implementation` post-load when using auto"],"tags":["v1","flash-attention","batching","configuration"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}