hiyouga/LlamaFactory · error · ValueError

Unsupported dummy media modality

Error message

Unsupported dummy media modality: {modality!r} (expected image/video/audio).

What it means

Renderer.get_dummy_media_fragment(modality) only accepts 'image', 'video', or 'audio'. Any other string (typos like 'img', 'Audio', or future modalities) is rejected immediately with the allowed set echoed in the message.

Solutions

1. Pass one of exactly 'image', 'video', 'audio' (lowercase)
2. Map content block types before calling: 'image_url'->'image', 'video_url'->'video', 'audio_url'->'audio'
3. Validate the modality against the allowed set at the call site for a clearer upstream error

Example fix

# before
frag = renderer.get_dummy_media_fragment(block["type"])  # 'image_url' -> ValueError

# after
_MOD = {"image_url": "image", "video_url": "video", "audio_url": "audio"}
frag = renderer.get_dummy_media_fragment(_MOD[block["type"]])

def is_supported_modality(m: str) -> bool:
    return m in ("image", "video", "audio")

When it happens

Trigger: Calling get_dummy_media_fragment with a modality string outside {'image','video','audio'} — commonly a typo, a caller-derived modality name, or case mismatch ('Image').

Common situations: Generic multimodal plumbing that derives the modality from content block types like 'image_url' and passes it unnormalized.

Related errors

• Cannot build a dummy media fragment for a text-only…
• placeholder count ( ) != number of blocks ( ); media must…
• Processor did not emit
• tool_call must be a JSON object with 'name' and 'arguments'…
• tool_call value is not valid JSON

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/16bd2d4a320a9f8b. Report an issue: GitHub.

Appendix: source

Thrown at src/llamafactory/v1/core/rendering/rendering.py:211

        Args:
            messages: The messages to render. For training the last message must be the supervised
                assistant turn (use ``process_samples`` to split multi-turn conversations).
            tools: JSON string of tool definitions.
            is_generate: Whether to render for generation (adds generation prompt, no supervision).
            **kwargs: Extra chat-template kwargs (e.g. ``enable_thinking``) forwarded verbatim to
                ``apply_chat_template``; unset ones fall back to the template's own defaults. A
                supervised assistant turn carrying reasoning forces ``enable_thinking=True``.

        Returns:
            ModelInput with input_ids, attention_mask, labels, and loss_weights.
        """
        return _render_messages(self.processor, messages, tools, is_generate, **kwargs)

    def get_dummy_media_fragment(self, modality: str) -> dict:
        """Build (and cache) a minimal valid media fragment for ``modality`` ("image"|"video"|"audio")."""
        if modality not in ("image", "video", "audio"):
            raise ValueError(f"Unsupported dummy media modality: {modality!r} (expected image/video/audio).")
        if is_tokenizer(self.processor):
            raise RuntimeError("Cannot build a dummy media fragment for a text-only processor.")

        if not hasattr(self, "_dummy_fragments"):
            self._dummy_fragments: dict[str, dict] = {}
        if modality in self._dummy_fragments:
            return self._dummy_fragments[modality]

        from PIL import Image as _PILImage

        if modality == "image":
            media_block = {"type": "image_url", "value": _PILImage.new("RGB", (64, 64))}
            target, presence_key = 1, "pixel_values"
        elif modality == "video":
            # A minimal clip: the temporal patch size is typically 2, so provide two frames.
            media_block = {"type": "video_url", "value": np.zeros((2, 64, 64, 3), dtype=np.uint8)}
            target, presence_key = 2, "pixel_values_videos"
        else:

View on GitHub (pinned to f28afaf635)