hiyouga/LlamaFactory · error · ValueError

MOSS-VL encountered nested video token blocks after tokeniza

Error message

MOSS-VL encountered nested video token blocks after tokenization. Please increase `cutoff_len` if a video placeholder was truncated.

What it means

MOSS-VL plugin post-validates tokenized input_ids: after the tokenizer expands placeholders, a vision_start token encountered while already inside a video block means two video blocks overlap — i.e. the token stream is corrupted, almost always because cutoff_len truncated a video placeholder mid-block, leaving an unterminated start token adjacent to the next media block.

Source

Thrown at src/llamafactory/data/mm_plugin.py:572

        return dict(processor.video_processor(videos=videos, **video_kwargs))

    def _get_media_order_from_ids(
        self,
        input_ids: list[int],
        processor: "MMProcessor",
        num_images: int,
        num_videos: int,
        expected_video_frames: Optional[list[int]] = None,
    ) -> list[str]:
        media_order = []
        video_frame_counts = []
        in_video = False
        current_video_frames = 0
        for token_id in input_ids:
            if token_id == processor.vision_start_token_id:
                if in_video:
                    raise ValueError(
                        "MOSS-VL encountered nested video token blocks after tokenization. "
                        "Please increase `cutoff_len` if a video placeholder was truncated."
                    )

                media_order.append("video")
                in_video = True
                current_video_frames = 0
            elif token_id == processor.vision_end_token_id:
                if not in_video:
                    raise ValueError(
                        "MOSS-VL encountered a video end token without a matching start token after tokenization. "
                        "Please increase `cutoff_len` if a video placeholder was truncated."
                    )

                video_frame_counts.append(current_video_frames)
                in_video = False
            elif token_id == processor.image_token_id:
                if in_video:

View on GitHub (pinned to f28afaf635)

Solutions

Increase cutoff_len in the training/inference YAML so a full video placeholder plus text fits.
Reduce video_maxlen (or video_fps) so each video expands to fewer tokens.
Shorten the text portion of affected samples.

Example fix

### before
cutoff_len: 2048
video_maxlen: 256
### after
cutoff_len: 8192
video_maxlen: 64

Defensive patterns

Strategy: validation

Validate before calling

# estimate worst-case tokens before training
frames = video_maxlen
tokens_per_frame = 196  # MOSS-VL tiles; use your measured value
est = frames * tokens_per_frame + 32
assert cutoff_len > est + 512, 'raise cutoff_len or lower video_maxlen'

Prevention

Always pair MOSS-VL video configs with a cutoff_len comfortably larger than max placeholder expansion.
Start with small video_maxlen, raise only after a successful dry run.

When it happens

Trigger: Training/chat with MOSS-VL where cutoff_len is smaller than the expanded video placeholder (vision_bos + N image tokens + vision_eos), so truncation cuts off the end token and the next sample's/media's start token appears inside the still-open block.

Common situations: Long videos with high frame counts (video_maxlen large) plus a modest cutoff_len (e.g. 2048); sharing a config across models where MOSS-VL needs far more tokens per video than the model it was copied from.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/b29658eaac9755bf. Report an issue: GitHub.