hiyouga/LlamaFactory · error · RuntimeError

Cannot build a dummy media fragment for a text-only processo

Error message

Cannot build a dummy media fragment for a text-only processor.

What it means

get_dummy_media_fragment is only meaningful for a multimodal processor (one with a feature extractor / image processor). If the renderer was constructed with a bare tokenizer (is_tokenizer(processor) is True), there is no way to produce valid pixel/audio tensors, so a RuntimeError is raised.

Source

Thrown at src/llamafactory/v1/core/rendering/rendering.py:213

            messages: The messages to render. For training the last message must be the supervised
                assistant turn (use ``process_samples`` to split multi-turn conversations).
            tools: JSON string of tool definitions.
            is_generate: Whether to render for generation (adds generation prompt, no supervision).
            **kwargs: Extra chat-template kwargs (e.g. ``enable_thinking``) forwarded verbatim to
                ``apply_chat_template``; unset ones fall back to the template's own defaults. A
                supervised assistant turn carrying reasoning forces ``enable_thinking=True``.

        Returns:
            ModelInput with input_ids, attention_mask, labels, and loss_weights.
        """
        return _render_messages(self.processor, messages, tools, is_generate, **kwargs)

    def get_dummy_media_fragment(self, modality: str) -> dict:
        """Build (and cache) a minimal valid media fragment for ``modality`` ("image"|"video"|"audio")."""
        if modality not in ("image", "video", "audio"):
            raise ValueError(f"Unsupported dummy media modality: {modality!r} (expected image/video/audio).")
        if is_tokenizer(self.processor):
            raise RuntimeError("Cannot build a dummy media fragment for a text-only processor.")

        if not hasattr(self, "_dummy_fragments"):
            self._dummy_fragments: dict[str, dict] = {}
        if modality in self._dummy_fragments:
            return self._dummy_fragments[modality]

        from PIL import Image as _PILImage

        if modality == "image":
            media_block = {"type": "image_url", "value": _PILImage.new("RGB", (64, 64))}
            target, presence_key = 1, "pixel_values"
        elif modality == "video":
            # A minimal clip: the temporal patch size is typically 2, so provide two frames.
            media_block = {"type": "video_url", "value": np.zeros((2, 64, 64, 3), dtype=np.uint8)}
            target, presence_key = 2, "pixel_values_videos"
        else:
            # A short synthetic waveform at the model's sampling rate; the feature extractor pads it.
            sr = self.processor.feature_extractor.sampling_rate

View on GitHub (pinned to f28afaf635)

Solutions

1. Gate the call on modality: only request dummy fragments when the model is multimodal (processor is not a plain tokenizer)
2. Use the renderer's is_tokenizer-safe check or hasattr(processor, 'feature_extractor')/'image_processor' before calling
3. For text-only models, pad with pure text fragments instead

Example fix

# before
frag = renderer.get_dummy_media_fragment("image")  # renderer built from a tokenizer

# after
from llamafactory.v1.core.rendering.rendering import is_tokenizer
frag = None if is_tokenizer(renderer.processor) else renderer.get_dummy_media_fragment("image")

def renderer_supports_media(renderer) -> bool:
    return not is_tokenizer(renderer.processor)  # multimodal processor present

When it happens

Trigger: Building the renderer with tokenizer-only (text model) and then calling get_dummy_media_fragment for any modality — typically in generic padding/batching code that unconditionally requests dummy fragments.

Common situations: Shared collation code reused across text-only and multimodal models; forgetting to gate the dummy-fragment path on model modality.

Related errors

• Processor was not found, please check and update your model
• Unsupported dummy media modality: {modality!r} (expected ima
• Processor did not emit {modality} placeholder tokens for the
• Please upgrade `transformers` to 4.34.0
• Unable to process key {key}

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/55229facaf586b84. Report an issue: GitHub.