{"record":{"id":"81db4f3969cc92b6","repo":"hiyouga/LlamaFactory","slug":"neat-packing-is-not-supported-for-gemma4-gpt-oss","errorCode":null,"errorMessage":"Neat packing is not supported for gemma4, gpt_oss models for now.","messagePattern":"Neat packing is not supported for gemma4, gpt_oss models for now\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/collator.py","lineNumber":504,"sourceCode":"            return {\"data\": features, \"input_ids\": features[\"input_ids\"], \"labels\": features[\"labels\"]}\n\n        return features\n\n\n@dataclass\nclass SFTDataCollatorWith4DAttentionMask(MultiModalDataCollatorForSeq2Seq):\n    r\"\"\"Data collator for 4d attention mask.\"\"\"\n\n    block_diag_attn: bool = False\n    attn_implementation: Literal[\"eager\", \"sdpa\", \"flash_attention_2\"] = \"eager\"\n    compute_dtype: \"torch.dtype\" = torch.float32\n    neat_packing: bool = False\n\n    def __post_init__(self):\n        super().__post_init__()\n        if self.neat_packing and self.attn_implementation == \"flash_attention_2\":\n            if self.model is not None and getattr(self.model.config, \"model_type\", None) in [\"gemma4\", \"gpt_oss\"]:\n                raise ValueError(\"Neat packing is not supported for gemma4, gpt_oss models for now.\")\n\n    @staticmethod\n    def _unpad_packed_features(features: dict[str, Any]) -> None:\n        r\"\"\"Trim padded positions for packed FA2 batches.\"\"\"\n        attention_mask = features.get(\"attention_mask\")\n        if not torch.is_tensor(attention_mask) or attention_mask.dim() != 2 or attention_mask.size(0) != 1:\n            return\n\n        seq_len = attention_mask.size(1)\n        non_padding_indices = torch.nonzero(attention_mask[0] != 0, as_tuple=False).flatten()\n        if non_padding_indices.numel() == seq_len:\n            return\n\n        keys_on_seq_dim_1 = {\"input_ids\", \"labels\", \"attention_mask\", \"token_type_ids\"}\n        for key, value in list(features.items()):\n            if not torch.is_tensor(value):\n                continue\n","sourceCodeStart":486,"sourceCodeEnd":522,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/collator.py#L486-L522","documentation":"SFTDataCollatorWith4DAttentionMask.__post_init__ raises ValueError when neat_packing is enabled together with attn_implementation='flash_attention_2' on gemma4 or gpt_oss models. These architectures build attention in a way incompatible with the 4D block-diagonal mask trick neat packing relies on under FA2, so the combination is rejected up front.","triggerScenarios":"Building the SFT collator with neat_packing=True and attn_implementation='flash_attention_2' while model.config.model_type is 'gemma4' or 'gpt_oss' — in practice, a training YAML with neat_packing: true and flash_attention2: true for those models.","commonSituations":"Reusing a tuned FA2+neat-packing config across model families; enabling neat_packing globally in a shared config base that gemma4/gpt_oss runs inherit.","solutions":["Set neat_packing: false for gemma4/gpt_oss runs, or","Switch attn_implementation to sdpa/eager for those models (drop flash_attention_2).","Model-conditionalize the packing flags in your config pipeline."],"exampleFix":"# before (yaml)\nflash_attention_2: true\nneat_packing: true\n\n# after (yaml, for gemma4/gpt_oss)\nflash_attention_2: true\nneat_packing: false","handlingStrategy":"validation","validationCode":"if getattr(model.config, \"model_type\", None) in {\"gemma4\", \"gpt_oss\"}:\n    assert not (neat_packing and attn_implementation == \"flash_attention_2\"), \\\n        \"neat_packing + FA2 unsupported for gemma4/gpt_oss\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate the (model_type, neat_packing, attn_implementation) triple in a config preflight hook.","Keep a per-model defaults file rather than one global 'performance' preset."],"tags":["neat-packing","flash-attention","gemma","gpt-oss","training"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}