{"record":{"id":"84fbf921b39753f7","repo":"hiyouga/LlamaFactory","slug":"omni-models-are-not-supported-for-packed-sequences","errorCode":null,"errorMessage":"Omni models are not supported for packed sequences for now.","messagePattern":"Omni models are not supported for packed sequences for now\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/collator.py","lineNumber":445,"sourceCode":"            \"qwen2_5_omni_thinker\",\n            \"qwen3_omni_moe_thinker\",\n        ]\n\n        if self.get_rope_func is not None:\n            # for mmrope situation, we should calculate position_ids and rope_deltas per sample.\n            # When neat_packing is on, each sample has packing_params; None means no packing for that sample.\n            boundaries_list = [p.get(\"sequence_boundaries\") if p is not None else None for p in packing_params_list]\n            has_packing = any(b is not None and len(b) > 2 for b in boundaries_list)\n            if has_dummy_image and has_packing:\n                # FIXME: too tricky, need to be refactored @kuangdd\n                features[\"has_dummy_image\"] = True\n\n            # When fake image/audio was injected, sequence_boundaries no longer match the tensor; use non-packing path.\n            if not has_packing:\n                self._compute_rope_position_ids(features, mm_inputs)\n            else:\n                if is_omni:  # TODO: support omni models for packed sequences @kuangdd\n                    raise RuntimeError(\"Omni models are not supported for packed sequences for now.\")\n\n                self._compute_rope_position_ids_with_packing(\n                    features,\n                    mm_inputs,\n                    packing_params_list,\n                    batch_imglens,\n                    batch_vidlens,\n                    batch_audlens,\n                    has_dummy_image,\n                )\n\n            # For transformers compatibility, after https://github.com/huggingface/transformers/issues/39400\n            if features[\"position_ids\"].dim() == 3:\n                features[\"position_ids\"] = torch.cat(\n                    [features[\"position_ids\"][0].unsqueeze(0), features[\"position_ids\"]], dim=0\n                )\n\n        if (","sourceCodeStart":427,"sourceCodeEnd":463,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/collator.py#L427-L463","documentation":"In the multimodal collator's __call__, when packed sequences are detected (any sample's packing_params has >2 sequence boundaries) and the model is an omni model (audio+text like qwen2.5-omni), a RuntimeError is raised: the rope position-id computation for packed sequences (_compute_rope_position_ids_with_packing) is not implemented for omni inputs.","triggerScenarios":"Training an omni model (e.g. qwen2.5-omni) with packing enabled (neat_packing or default packing) so that collator batches carry packing_params, while the batch also contains multimodal inputs routed through the omni path.","commonSituations":"Copying a packing-enabled SFT config (written for a text or vision model) to fine-tune an audio-capable omni model; enabling efficient/packing defaults in a shared base config that omni runs also inherit.","solutions":["Set packing: false (and neat_packing: false if set) in the training YAML for omni models.","If you need packing for omni, wait for/watch the upstream TODO (support omni models for packed sequences) or contribute the omni branch in _compute_rope_position_ids_with_packing.","Split the omni dataset out of mixed packing-enabled runs."],"exampleFix":"# before (yaml)\npacking: true\nneat_packing: true\n\n# after (yaml)\npacking: false","handlingStrategy":"validation","validationCode":"OMNI_TYPES = {\"qwen2_5_omni\"}  # extend as needed\nif getattr(model.config, \"model_type\", \"\") in OMNI_TYPES:\n    assert not data_args.packing and not training_args.neat_packing, \"omni models cannot train with packing\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep per-model-family config files so packing flags do not leak into omni runs.","Add a config-lint step that rejects packing:true for omni model paths."],"tags":["packing","omni","collator","training","audio"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}