hiyouga/LlamaFactory · error · ValueError

`padding_free` requires `flash_attn: flash_attention_2`.

Error message

`padding_free` requires `flash_attn: flash_attention_2`.

What it means

v1 `BaseTrainer._create_batch_generator` rejects `batching_strategy: padding_free` unless the model's attention implementation is `flash_attention_2`. Padding-free batching packs sequences of different lengths into one varlen batch, which only flash-attention's varlen kernel can score correctly; padded SDPA/eager batches would mix tokens across sequences.

Source

Thrown at src/llamafactory/v1/core/base_trainer.py:169

            if model.config.model_type == "qwen3_5":
                raise RuntimeError(
                    "Sequence parallel is not supported for qwen3.5 model due to its different attention implementation, which will be supported in the future."
                )
            from ..plugins.model_plugins.parallelization.sequence_parallel import SequenceParallelModelPlugin

            if model.config._attn_implementation != "flash_attention_2":
                raise ValueError(
                    "Sequence parallelism requires flash attention. Please set `flash_attn: flash_attention_2`."
                )

            SequenceParallelModelPlugin(self.args.cp_mode)(model, self.args.cp_size)

    def _create_batch_generator(self) -> None:
        if (
            self.args.batching_strategy == BatchingStrategy.PADDING_FREE
            and getattr(self.model.config, "_attn_implementation", None) != "flash_attention_2"
        ):
            raise ValueError("`padding_free` requires `flash_attn: flash_attention_2`.")

        self.train_batch_generator = BatchGenerator(
            dataset=self.train_dataset,
            renderer=self.renderer,
            micro_batch_size=self.args.micro_batch_size,
            global_batch_size=self.args.global_batch_size,
            cutoff_len=self.args.cutoff_len,
            batching_workers=self.args.batching_workers,
            batching_strategy=self.args.batching_strategy,
            seed=self.args.seed,
        )

    def _shard_model(self) -> None:
        if self.args.dist_config is None:
            if DistributedInterface().get_world_size(Dim.DP) > 1:
                from torch.nn.parallel import DistributedDataParallel as DDP

                logger.warning_rank0(

View on GitHub (pinned to f28afaf635)

Solutions

1. Set `flash_attn: flash_attention_2` in model args and install flash-attn for your torch/CUDA combo
2. If flash-attn cannot be installed, use a padding or packing strategy that does not require FA2
3. Verify the resolved implementation after model load (log `model.config._attn_implementation`) if `auto` is used

Example fix

# before (yaml)
model:
  flash_attn: sdpa
training:
  batching_strategy: padding_free

# after (yaml)
model:
  flash_attn: flash_attention_2
training:
  batching_strategy: padding_free

def validate_padding_free(flash_attn: str, strategy: str) -> None:
    if strategy == "padding_free" and flash_attn != "flash_attention_2":
        raise SystemExit("padding_free requires flash_attn: flash_attention_2")

When it happens

Trigger: Setting `batching_strategy: padding_free` while the model runs with sdpa/eager attention (flash-attn missing, `flash_attn: auto` fell back, or explicitly set to a non-FA2 value).

Common situations: CPU or older-GPU environments where flash-attn is not installed; enabling packing/padding-free to speed up training without checking kernel availability.

Related errors

• `dynamic_batching` requires `max_steps` because it is step-d
• `save_epochs` is not supported with `dynamic_batching`; use
• Sequence parallelism requires flash attention. Please set `f
• world_size ({helper.get_world_size()}) must be divisible by
• mp_replicate_size * mp_shard_size must equal to world_size,

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/e6e76cb04647c64b. Report an issue: GitHub.