{"record":{"id":"af9e7e121732d193","repo":"hiyouga/LlamaFactory","slug":"template-is-required-for-multimodaldatacollator","errorCode":null,"errorMessage":"Template is required for MultiModalDataCollator.","messagePattern":"Template is required for MultiModalDataCollator\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/collator.py","lineNumber":148,"sourceCode":"    attention_mask_4d = (indices == indices_t) & non_padding_mask & tril_mask\n    # Invert the attention mask.\n    attention_mask_4d = torch.where(attention_mask_4d, zero_tensor, min_dtype)\n    return attention_mask_4d\n\n\n@dataclass\nclass MultiModalDataCollatorForSeq2Seq(DataCollatorForSeq2Seq):\n    r\"\"\"Data collator that supports VLMs.\n\n    Features should contain input_ids, attention_mask, labels, and optionally contain images, videos and audios.\n    \"\"\"\n\n    template: Optional[\"Template\"] = None\n    processor: Optional[\"ProcessorMixin\"] = None\n\n    def __post_init__(self):\n        if self.template is None:\n            raise ValueError(\"Template is required for MultiModalDataCollator.\")\n\n        if isinstance(self.model, PeftModel):\n            self.model = self.model.base_model.model\n\n        if getattr(getattr(self.model, \"config\", None), \"model_type\", None) == \"moss_vl\":\n            self.get_rope_func = None  # MOSS-VL computes its own XRoPE positions in model.forward.\n        elif self.model is not None and hasattr(self.model, \"get_rope_index\"):  # for qwen2vl mrope\n            self.get_rope_func = self.model.get_rope_index  # transformers < 4.52.0 or qwen2.5 omni\n        elif self.model is not None and hasattr(self.model, \"model\") and hasattr(self.model.model, \"get_rope_index\"):\n            self.get_rope_func = self.model.model.get_rope_index  # transformers >= 4.52.0\n        else:\n            self.get_rope_func = None\n\n    def _compute_rope_position_ids(self, features: dict[str, \"torch.Tensor\"], mm_inputs: dict[str, Any]) -> None:\n        r\"\"\"Compute position_ids and rope_deltas via get_rope_func for VLMs.\"\"\"\n        rope_index_kwargs = {\n            \"input_ids\": features[\"input_ids\"],\n            \"image_grid_thw\": mm_inputs.get(\"image_grid_thw\"),","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/collator.py#L130-L166","documentation":"MultiModalDataCollatorForSeq2Seq.__post_init__ raises ValueError when the dataclass is instantiated with template=None. The template is load-bearing: it supplies the mm_plugin used to preprocess images/videos/audios into model inputs, so a collator without one cannot build multimodal features.","triggerScenarios":"Constructing MultiModalDataCollatorForSeq2Seq(tokenizer=..., model=...) without template=..., typically when hand-rolling a Trainer instead of going through get_template_and_fix_tokenizer, or when a code change drops the template argument.","commonSituations":"Custom training scripts that copy the HF DataCollatorForSeq2Seq constructor signature; refactors that pass template through a kwargs dict which silently swallows the key; template lookup returning None due to a typo in the template name.","solutions":["Pass the template produced by get_template_and_fix_tokenizer(tokenizer, model_args, data_args, ...) when building the collator.","Check the template name in your dataset config exists in the TEMPLATES registry so lookup does not yield None.","If you truly need a text-only collator, use the plain DataCollatorForSeq2Seq instead of the multimodal subclass."],"exampleFix":"# before\ncollator = MultiModalDataCollatorForSeq2Seq(tokenizer=tokenizer, model=model)\n\n# after\ntemplate = get_template_and_fix_tokenizer(tokenizer, model_args, data_args)\ncollator = MultiModalDataCollatorForSeq2Seq(tokenizer=tokenizer, model=model, template=template)","handlingStrategy":"validation","validationCode":"template = get_template_and_fix_tokenizer(tokenizer, model_args, data_args)\nif template is None:\n    raise ValueError(f\"unknown template: {data_args.template}\")\ncollator = MultiModalDataCollatorForSeq2Seq(tokenizer=tokenizer, model=model, template=template)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always source the template from get_template_and_fix_tokenizer; never hand-construct the collator without it.","Assert template is not None right after lookup to fail before training starts."],"tags":["collator","multimodal","config","training"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}