hiyouga/LlamaFactory · error · ValueError

Template is required for MultiModalDataCollator.

Error message

Template is required for MultiModalDataCollator.

What it means

MultiModalDataCollatorForSeq2Seq.__post_init__ raises ValueError when the dataclass is instantiated with template=None. The template is load-bearing: it supplies the mm_plugin used to preprocess images/videos/audios into model inputs, so a collator without one cannot build multimodal features.

Source

Thrown at src/llamafactory/data/collator.py:148

    attention_mask_4d = (indices == indices_t) & non_padding_mask & tril_mask
    # Invert the attention mask.
    attention_mask_4d = torch.where(attention_mask_4d, zero_tensor, min_dtype)
    return attention_mask_4d


@dataclass
class MultiModalDataCollatorForSeq2Seq(DataCollatorForSeq2Seq):
    r"""Data collator that supports VLMs.

    Features should contain input_ids, attention_mask, labels, and optionally contain images, videos and audios.
    """

    template: Optional["Template"] = None
    processor: Optional["ProcessorMixin"] = None

    def __post_init__(self):
        if self.template is None:
            raise ValueError("Template is required for MultiModalDataCollator.")

        if isinstance(self.model, PeftModel):
            self.model = self.model.base_model.model

        if getattr(getattr(self.model, "config", None), "model_type", None) == "moss_vl":
            self.get_rope_func = None  # MOSS-VL computes its own XRoPE positions in model.forward.
        elif self.model is not None and hasattr(self.model, "get_rope_index"):  # for qwen2vl mrope
            self.get_rope_func = self.model.get_rope_index  # transformers < 4.52.0 or qwen2.5 omni
        elif self.model is not None and hasattr(self.model, "model") and hasattr(self.model.model, "get_rope_index"):
            self.get_rope_func = self.model.model.get_rope_index  # transformers >= 4.52.0
        else:
            self.get_rope_func = None

    def _compute_rope_position_ids(self, features: dict[str, "torch.Tensor"], mm_inputs: dict[str, Any]) -> None:
        r"""Compute position_ids and rope_deltas via get_rope_func for VLMs."""
        rope_index_kwargs = {
            "input_ids": features["input_ids"],
            "image_grid_thw": mm_inputs.get("image_grid_thw"),

View on GitHub (pinned to f28afaf635)

Solutions

  1. Pass the template produced by get_template_and_fix_tokenizer(tokenizer, model_args, data_args, ...) when building the collator.
  2. Check the template name in your dataset config exists in the TEMPLATES registry so lookup does not yield None.
  3. If you truly need a text-only collator, use the plain DataCollatorForSeq2Seq instead of the multimodal subclass.

Example fix

# before
collator = MultiModalDataCollatorForSeq2Seq(tokenizer=tokenizer, model=model)

# after
template = get_template_and_fix_tokenizer(tokenizer, model_args, data_args)
collator = MultiModalDataCollatorForSeq2Seq(tokenizer=tokenizer, model=model, template=template)
Defensive patterns

Strategy: validation

Validate before calling

template = get_template_and_fix_tokenizer(tokenizer, model_args, data_args)
if template is None:
    raise ValueError(f"unknown template: {data_args.template}")
collator = MultiModalDataCollatorForSeq2Seq(tokenizer=tokenizer, model=model, template=template)

Prevention

When it happens

Trigger: Constructing MultiModalDataCollatorForSeq2Seq(tokenizer=..., model=...) without template=..., typically when hand-rolling a Trainer instead of going through get_template_and_fix_tokenizer, or when a code change drops the template argument.

Common situations: Custom training scripts that copy the HF DataCollatorForSeq2Seq constructor signature; refactors that pass template through a kwargs dict which silently swallows the key; template lookup returning None due to a typo in the template name.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/af9e7e121732d193. Report an issue: GitHub.