{"record":{"id":"f2c23b041be46ecf","repo":"hiyouga/LlamaFactory","slug":"moss-vl-video-frame-tokens-do-not-match-the-proces","errorCode":null,"errorMessage":"MOSS-VL video frame tokens do not match the processed video after tokenization: tokens={video_frame_counts}, frames={expected_video_frames}. Please increase `cutoff_len` or reduce `video_maxlen`.","messagePattern":"MOSS-VL video frame tokens do not match the processed video after tokenization: tokens=(.+?), frames=(.+?)\\. Please increase `cutoff_len` or reduce `video_maxlen`\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/mm_plugin.py","lineNumber":609,"sourceCode":"                    current_video_frames += 1\n                else:\n                    media_order.append(\"image\")\n\n        if in_video:\n            raise ValueError(\n                \"MOSS-VL encountered an incomplete video token block after tokenization. \"\n                \"Please increase `cutoff_len` or reduce `video_maxlen`.\"\n            )\n\n        if media_order.count(\"image\") != num_images or media_order.count(\"video\") != num_videos:\n            raise ValueError(\n                \"MOSS-VL media tokens do not match the provided media after tokenization: \"\n                f\"order={media_order}, images={num_images}, videos={num_videos}. \"\n                \"Please increase `cutoff_len` if a visual placeholder was truncated.\"\n            )\n\n        if expected_video_frames is not None and video_frame_counts != expected_video_frames:\n            raise ValueError(\n                \"MOSS-VL video frame tokens do not match the processed video after tokenization: \"\n                f\"tokens={video_frame_counts}, frames={expected_video_frames}. \"\n                \"Please increase `cutoff_len` or reduce `video_maxlen`.\"\n            )\n\n        return media_order\n\n    @override\n    def process_messages(\n        self,\n        messages: list[dict[str, str]],\n        images: list[\"ImageInput\"],\n        videos: list[\"VideoInput\"],\n        audios: list[\"AudioInput\"],\n        processor: Optional[\"MMProcessor\"],\n    ) -> list[dict[str, str]]:\n        self._validate_input(processor, images, videos, audios)\n        self._validate_messages(messages, images, videos, audios)","sourceCodeStart":591,"sourceCodeEnd":627,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/mm_plugin.py#L591-L627","documentation":"MOSS-VL post-tokenization check: the number of image tokens counted inside each reconstructed video block differs from expected_video_frames (the frame counts of the processed videos). Truncation inside a video block, or a frame-count mismatch between preprocessing and tokenization, produces different frame token counts.","triggerScenarios":"video placeholder truncated partway (some frame tokens cut), or the processor resamples videos differently than the plugin expected — e.g. video_maxlen / fps applied inconsistently between the regularize step and the tokenizer expansion.","commonSituations":"cutoff_len just barely fits a video so a few frame tokens get cut; mixing plugin kwargs (video_fps, video_maxlen) with processor defaults that disagree.","solutions":["Increase cutoff_len or reduce video_maxlen so every frame token of every video fits.","Pass consistent video_fps/video_maxlen kwargs to get_mm_plugin so regularization and tokenization agree.","Check for duplicate <video> tags mapping to fewer actual videos (count mismatch upstream)."],"exampleFix":"### before\nplugin = get_mm_plugin(name='moss_vl', video_maxlen=256)\ncutoff_len: 4096\n### after\nplugin = get_mm_plugin(name='moss_vl', video_maxlen=64)\ncutoff_len: 16384","handlingStrategy":"validation","validationCode":"# verify each video block contains the expected frame token count\nin_video, count, counts = False, 0, []\nfor t in ids:\n    if t == processor.vision_start_token_id: in_video, count = True, 0\n    elif t == processor.vision_end_token_id: counts.append(count); in_video = False\n    elif t == processor.image_token_id and in_video: count += 1\nassert counts == expected_video_frames, 'frame tokens truncated; raise cutoff_len or reduce video_maxlen'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pass video_fps/video_maxlen explicitly to get_mm_plugin so they match the processor config.","Dry-run one batch; MOSS-VL fails fast at tokenization, which is cheap to test."],"tags":["multimodal","moss-vl","video","frames","truncation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}