{"record":{"id":"5a44e12f7b5e6dff","repo":"hiyouga/LlamaFactory","slug":"self-model-config-model-type-requires-3d-positio","errorCode":null,"errorMessage":"{self.model.config.model_type} requires 3D position ids for mrope.","messagePattern":"(.+?) requires 3D position ids for mrope\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/collator.py","lineNumber":468,"sourceCode":"                    packing_params_list,\n                    batch_imglens,\n                    batch_vidlens,\n                    batch_audlens,\n                    has_dummy_image,\n                )\n\n            # For transformers compatibility, after https://github.com/huggingface/transformers/issues/39400\n            if features[\"position_ids\"].dim() == 3:\n                features[\"position_ids\"] = torch.cat(\n                    [features[\"position_ids\"][0].unsqueeze(0), features[\"position_ids\"]], dim=0\n                )\n\n        if (\n            self.model is not None\n            and getattr(self.model.config, \"model_type\", None) in MROPE_MODELS\n            and (\"position_ids\" not in features or features[\"position_ids\"].dim() != 3)\n        ):\n            raise ValueError(f\"{self.model.config.model_type} requires 3D position ids for mrope.\")\n\n        if (\n            \"cross_attention_mask\" in mm_inputs and mm_inputs[\"cross_attention_mask\"].dtype != torch.bool\n        ):  # for mllama inputs when pad_to_multiple_of is enabled\n            cross_attention_mask = mm_inputs.pop(\"cross_attention_mask\")\n            seq_len = features[\"input_ids\"].size(1)\n            orig_len = cross_attention_mask.size(1)\n            mm_inputs[\"cross_attention_mask\"] = F.pad(cross_attention_mask, (0, 0, 0, 0, 0, seq_len - orig_len))\n\n        if is_moss_vl:\n            mm_inputs = self.template.mm_plugin.post_process_mossvl_inputs(features, mm_inputs, self.processor)\n\n        features.update(mm_inputs)\n\n        if \"image_bound\" in features:  # for minicpmv inputs\n            bsz, seq_length = features[\"input_ids\"].shape\n            features[\"position_ids\"] = torch.arange(seq_length).long().repeat(bsz, 1)\n            return {\"data\": features, \"input_ids\": features[\"input_ids\"], \"labels\": features[\"labels\"]}","sourceCodeStart":450,"sourceCodeEnd":486,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/collator.py#L450-L486","documentation":"After collation, if the model's config.model_type is in MROPE_MODELS (qwen2-vl family using 3D mrope position ids) but features lacks position_ids or its dim is not 3, the collator raises ValueError. The 3D tensor (3, batch, seq) encodes temporal/height/width positions required by mrope attention; a 2D or missing position_ids means the mrope path never ran (e.g. rope func absent) or produced the wrong rank.","triggerScenarios":"Training/evaluating an MROPE_MODELS model where get_rope_func resolved to None (model wrapper lacking get_rope_index on both model and model.model), or a code path replaced position_ids with a 2D tensor before this check; also packing/dummy-image paths that bypassed mrope computation.","commonSituations":"transformers version change moving get_rope_index off the expected attribute (the collator probes model.get_rope_index then model.model.get_rope_index); wrapping the model in a custom module that hides the attribute; using a qwen2-vl template with a non-qwen2-vl checkpoint or vice versa.","solutions":["Confirm model_type of your checkpoint is what you expect (print model.config.model_type) and that it matches the template family.","Upgrade/downgrade transformers to a version supported by your LlamaFactory release so get_rope_index is found (probe order: model, then model.model).","Avoid wrapping the model in custom classes that shadow get_rope_index; expose or delegate the attribute.","Ensure the multimodal collator (not a plain text collator) is used for mrope models."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"from llamafactory.data.collator import MROPE_MODELS\nif getattr(model.config, \"model_type\", None) in MROPE_MODELS:\n    rope_ok = hasattr(model, \"get_rope_index\") or (hasattr(model, \"model\") and hasattr(model.model, \"get_rope_index\"))\n    assert rope_ok, \"mrope model but get_rope_index not found — check transformers version\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pin transformers to the version matrix tested by LlamaFactory for qwen2-vl family.","Do not wrap models in custom containers that hide get_rope_index.","Match template family to checkpoint family."],"tags":["mrope","collator","qwen-vl","multimodal","training"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}