hiyouga/LlamaFactory · error · ValueError

The number of videos does not match the number of {VIDEO_PLA

Error message

The number of videos does not match the number of {VIDEO_PLACEHOLDER} tokens in {messages}.

What it means

Thrown by BasePlugin._validate_messages during message preprocessing for any multimodal model. The number of video objects passed in the `videos` argument must equal the number of `<video>` (VIDEO_PLACEHOLDER) tokens embedded in the message content. LlamaFactory replaces each placeholder with one video, so a mismatch would desynchronize media and tokens.

Source

Thrown at src/llamafactory/data/mm_plugin.py:214

        r"""Validate if the number of images, videos and audios match the number of placeholders in messages."""
        num_image_tokens, num_video_tokens, num_audio_tokens = 0, 0, 0
        for message in messages:
            num_image_tokens += message["content"].count(IMAGE_PLACEHOLDER)
            num_video_tokens += message["content"].count(VIDEO_PLACEHOLDER)
            num_audio_tokens += message["content"].count(AUDIO_PLACEHOLDER)

        if len(images) != num_image_tokens:
            raise ValueError(
                f"The number of images does not match the number of {IMAGE_PLACEHOLDER} tokens in {messages}."
            )

        if len(videos) != num_video_tokens:
            raise ValueError(
                f"The number of videos does not match the number of {VIDEO_PLACEHOLDER} tokens in {messages}."
            )

        if len(audios) != num_audio_tokens:
            raise ValueError(
                f"The number of audios does not match the number of {AUDIO_PLACEHOLDER} tokens in {messages}."
            )

    def _preprocess_image(
        self, image: "ImageObject", image_max_pixels: int, image_min_pixels: int, **kwargs
    ) -> "ImageObject":
        r"""Pre-process a single image."""
        if (image.width * image.height) > image_max_pixels:
            resize_factor = math.sqrt(image_max_pixels / (image.width * image.height))
            width, height = int(image.width * resize_factor), int(image.height * resize_factor)
            image = image.resize((width, height))

        if (image.width * image.height) < image_min_pixels:
            resize_factor = math.sqrt(image_min_pixels / (image.width * image.height))
            width, height = int(image.width * resize_factor), int(image.height * resize_factor)
            image = image.resize((width, height))

        if image.mode != "RGB":

View on GitHub (pinned to f28afaf635)

Solutions

1. Count VIDEO_PLACEHOLDER occurrences in each sample's messages and make the `videos` list length match exactly (one video per <video> tag).
2. If the sample has no video, remove the <video> tag from the content string.
3. If a video column exists but the prompt lacks the tag, add <video> to the prompt (e.g. 'Describe the video: <video>').
4. Write a small preprocessing script that asserts content.count('<video>') == len(videos) for every row before training.

Example fix

// before
{"messages": [{"role": "user", "content": "Compare <video> and <video>."}], "videos": ["a.mp4"]}
// after
{"messages": [{"role": "user", "content": "Compare <video> and <video>."}], "videos": ["a.mp4", "b.mp4"]}

from llamafactory.data.mm_plugin import VIDEO_PLACEHOLDER

def check_sample(messages, videos):
    n = sum(m['content'].count(VIDEO_PLACEHOLDER) for m in messages if 'content' in m)
    assert n == len(videos), f'{n} <video> tags vs {len(videos)} videos'

When it happens

Trigger: Calling get_mm_plugin(...).process_messages(messages, images, videos, ...) or running training/inference where a sample's `messages` contain N `<video>` tags but the dataset row supplies a `videos` list of length != N. Typical rows: two `<video>` tags with one video file, or a video column but no `<video>` tag in the prompt.

Common situations: Hand-written JSON datasets where the prompt template and the media columns drift apart; converting a dataset from images to videos (or mixed samples) without updating the instruction text; extra whitespace or a typo like `<vid>` making the placeholder uncountable.

Related errors

• Number of videos ({len(videos)}) must match number of audios
• This model does not support video input. Please check whethe
• Expect input is a list of images, but got {type(image)}.
• Invalid image found in video frames.
• MOSS-VL encountered nested video token blocks after tokeniza

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/4275993a2d937764. Report an issue: GitHub.