hiyouga/LlamaFactory · error · ValueError

MOSS-VL media tokens do not match the provided media after t

Error message

MOSS-VL media tokens do not match the provided media after tokenization: order={media_order}, images={num_images}, videos={num_videos}. Please increase `cutoff_len` if a visual placeholder was truncated.

What it means

MOSS-VL post-tokenization check: after reconstructing the media order from input_ids, the counts of image and video blocks do not equal the num_images / num_videos passed in. Some placeholders were truncated away (or never expanded), so the model would attend to fewer media than provided.

Source

Thrown at src/llamafactory/data/mm_plugin.py:602

                        "Please increase `cutoff_len` if a video placeholder was truncated."
                    )

                video_frame_counts.append(current_video_frames)
                in_video = False
            elif token_id == processor.image_token_id:
                if in_video:
                    current_video_frames += 1
                else:
                    media_order.append("image")

        if in_video:
            raise ValueError(
                "MOSS-VL encountered an incomplete video token block after tokenization. "
                "Please increase `cutoff_len` or reduce `video_maxlen`."
            )

        if media_order.count("image") != num_images or media_order.count("video") != num_videos:
            raise ValueError(
                "MOSS-VL media tokens do not match the provided media after tokenization: "
                f"order={media_order}, images={num_images}, videos={num_videos}. "
                "Please increase `cutoff_len` if a visual placeholder was truncated."
            )

        if expected_video_frames is not None and video_frame_counts != expected_video_frames:
            raise ValueError(
                "MOSS-VL video frame tokens do not match the processed video after tokenization: "
                f"tokens={video_frame_counts}, frames={expected_video_frames}. "
                "Please increase `cutoff_len` or reduce `video_maxlen`."
            )

        return media_order

    @override
    def process_messages(
        self,
        messages: list[dict[str, str]],

View on GitHub (pinned to f28afaf635)

Solutions

Increase cutoff_len so every image/video placeholder survives tokenization.
Reduce the number of media per sample or their token footprint (smaller images, lower video_maxlen).
Verify placeholder counts in messages match the images/videos lists (guards the non-truncation path).

Example fix

### before
# 8 images per sample, cutoff_len 2048 -> placeholders truncated
cutoff_len: 2048
### after
cutoff_len: 16384

Defensive patterns

Strategy: validation

Validate before calling

order = []
for t in ids:
    if t == processor.vision_start_token_id: order.append('video')
    elif t == processor.image_token_id and (not order or order[-1] != 'video_open'): pass
# simpler: count blocks
n_img = sum(1 for t in ids if t == processor.image_token_id)
assert order.count('image') == expected_images and order.count('video') == expected_videos

Prevention

Keep media-per-sample counts modest so placeholders never approach cutoff_len.
Unit-test your dataset with the plugin's process_messages before a training run.

When it happens

Trigger: Calling MOSS-VL process_messages/process_token_ids where truncation removed entire media placeholders, or where placeholder counts in the text differ from the media lists (then _validate_messages usually fires first, but truncation bypasses it).

Common situations: cutoff_len smaller than the total media placeholder budget of a sample with several images/videos; samples with many images (image count high, each block large).

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/2a12319d72660381. Report an issue: GitHub.