hiyouga/LlamaFactory · error · ValueError

Neat packing is not supported for gemma4, gpt_oss models for

Error message

Neat packing is not supported for gemma4, gpt_oss models for now.

What it means

SFTDataCollatorWith4DAttentionMask.__post_init__ raises ValueError when neat_packing is enabled together with attn_implementation='flash_attention_2' on gemma4 or gpt_oss models. These architectures build attention in a way incompatible with the 4D block-diagonal mask trick neat packing relies on under FA2, so the combination is rejected up front.

Source

Thrown at src/llamafactory/data/collator.py:504

            return {"data": features, "input_ids": features["input_ids"], "labels": features["labels"]}

        return features


@dataclass
class SFTDataCollatorWith4DAttentionMask(MultiModalDataCollatorForSeq2Seq):
    r"""Data collator for 4d attention mask."""

    block_diag_attn: bool = False
    attn_implementation: Literal["eager", "sdpa", "flash_attention_2"] = "eager"
    compute_dtype: "torch.dtype" = torch.float32
    neat_packing: bool = False

    def __post_init__(self):
        super().__post_init__()
        if self.neat_packing and self.attn_implementation == "flash_attention_2":
            if self.model is not None and getattr(self.model.config, "model_type", None) in ["gemma4", "gpt_oss"]:
                raise ValueError("Neat packing is not supported for gemma4, gpt_oss models for now.")

    @staticmethod
    def _unpad_packed_features(features: dict[str, Any]) -> None:
        r"""Trim padded positions for packed FA2 batches."""
        attention_mask = features.get("attention_mask")
        if not torch.is_tensor(attention_mask) or attention_mask.dim() != 2 or attention_mask.size(0) != 1:
            return

        seq_len = attention_mask.size(1)
        non_padding_indices = torch.nonzero(attention_mask[0] != 0, as_tuple=False).flatten()
        if non_padding_indices.numel() == seq_len:
            return

        keys_on_seq_dim_1 = {"input_ids", "labels", "attention_mask", "token_type_ids"}
        for key, value in list(features.items()):
            if not torch.is_tensor(value):
                continue

View on GitHub (pinned to f28afaf635)

Solutions

1. Set neat_packing: false for gemma4/gpt_oss runs, or
2. Switch attn_implementation to sdpa/eager for those models (drop flash_attention_2).
3. Model-conditionalize the packing flags in your config pipeline.

Example fix

# before (yaml)
flash_attention_2: true
neat_packing: true

# after (yaml, for gemma4/gpt_oss)
flash_attention_2: true
neat_packing: false

if getattr(model.config, "model_type", None) in {"gemma4", "gpt_oss"}:
    assert not (neat_packing and attn_implementation == "flash_attention_2"), \
        "neat_packing + FA2 unsupported for gemma4/gpt_oss"

When it happens

Trigger: Building the SFT collator with neat_packing=True and attn_implementation='flash_attention_2' while model.config.model_type is 'gemma4' or 'gpt_oss' — in practice, a training YAML with neat_packing: true and flash_attention2: true for those models.

Common situations: Reusing a tuned FA2+neat-packing config across model families; enabling neat_packing globally in a shared config base that gemma4/gpt_oss runs inherit.

Related errors

• Unsupported model type: {getattr(config, 'model_type')}.
• Template is required for MultiModalDataCollator.
• Merged position_ids shape mismatch: got {features['position_
• Omni models are not supported for packed sequences for now.
• {self.model.config.model_type} requires 3D position ids for

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/81db4f3969cc92b6. Report an issue: GitHub.