hiyouga/LlamaFactory · error · NotImplementedError

batching_strategy={self.batching_strategy.value!r} does not

Error message

batching_strategy={self.batching_strategy.value!r} does not support multimodal data; use the NORMAL strategy for image/video training.

What it means

Non-NORMAL batching strategies (dynamic/packing/padding_free) collate ragged multimodal tensors with a bare default_collate and have no vision-tower alignment, so image/video/audio batches would crash or corrupt training. The scheduler detects any _MULTIMODAL_PASSTHROUGH_KEYS (pixel_values, input_features, etc.) in buffered samples under a non-NORMAL strategy and raises NotImplementedError instead of mishandling the data.

Source

Thrown at src/llamafactory/v1/core/utils/batching.py:328

                    samples: list[ModelInput] = next(self._data_iter)
                except StopIteration:
                    break

                self._buffer.put(self._drop_unsupervised(samples))
        else:
            from ...plugins.trainer_plugins.batching import BatchingPlugin

            BatchingPlugin(self.batching_strategy).fill_buffer(self._buffer, self._batch_info, self._next_samples)

    def _generate_batch(self) -> list[BatchInput] | None:
        if self.batching_strategy == BatchingStrategy.NORMAL:
            return default_collate_fn(self._buffer, self._batch_info, self.renderer)
        else:
            # Non-NORMAL strategies (dynamic / padding_free) collate ragged pixel tensors with a
            # bare default_collate and have no vision-tower alignment, so multimodal data would
            # crash or hang. Fail loud instead of silently mishandling it.
            if any(k in s for s in self._buffer.samples for k in _MULTIMODAL_PASSTHROUGH_KEYS):
                raise NotImplementedError(
                    f"batching_strategy={self.batching_strategy.value!r} does not support multimodal data; "
                    "use the NORMAL strategy for image/video training."
                )

            from ...plugins.trainer_plugins.batching import BatchingPlugin

            return BatchingPlugin(self.batching_strategy).generate_batch(self._buffer, self._batch_info)

    def _next_samples(self, restart: bool) -> list[ModelInput] | None:
        try:
            return next(self._data_iter)
        except StopIteration:
            if not restart:
                return None

            # Dynamic batching may restart the provider to fill one token-budgeted batch.
            self._data_iter = iter(self._data_provider)
            try:

View on GitHub (pinned to f28afaf635)

Solutions

1. Set batching_strategy to BatchingStrategy.NORMAL for the multimodal run
2. Keep separate config profiles: packing/padding_free for text models, NORMAL for image/video/audio models
3. Remove literal media placeholder text if media was unintentional (sample should not carry mm passthrough keys for pure-text training)

Example fix

# before
scheduler = Batching(..., batching_strategy=BatchingStrategy.PACKING)  # with image samples -> NotImplementedError

# after
from llamafactory.v1.core.utils.batching import BatchingStrategy
scheduler = Batching(..., batching_strategy=BatchingStrategy.NORMAL)

def strategy_allows_multimodal(strategy, samples) -> bool:
    if strategy == BatchingStrategy.NORMAL:
        return True
    return not any(k in s for s in samples for k in ("pixel_values", "input_features", "video_grid_thw"))

When it happens

Trigger: Training a multimodal model with batching_strategy set to a packing/dynamic/padding_free strategy while samples carry pixel_values/video/input_features keys (i.e. media was successfully rendered).

Common situations: Copying a text-model efficiency config (packing or padding_free for throughput) onto a vision/audio fine-tune; enabling efficient batching globally in a shared YAML that is also used for VLM runs.

Related errors

• Template is required for MultiModalDataCollator.
• Merged position_ids shape mismatch: got {features['position_
• Omni models are not supported for packed sequences for now.
• {self.model.config.model_type} requires 3D position ids for
• MOSS-VL batch metadata must have one entry per sample.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/f11baf3242a1f1f3. Report an issue: GitHub.