{"record":{"id":"a2ce0e64030b2c1a","repo":"hiyouga/LlamaFactory","slug":"number-of-videos-len-videos-must-match-number","errorCode":null,"errorMessage":"Number of videos ({len(videos)}) must match number of audios ({len(audios)}) when using audio in video.","messagePattern":"Number of videos \\((.+?)\\) must match number of audios \\((.+?)\\) when using audio in video\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/mm_plugin.py","lineNumber":3020,"sourceCode":"            video_grid_thw = [None] * len(videos)\n            audio_lengths = [None] * len(audios)\n\n        for message in messages:\n            content = message[\"content\"]\n            while IMAGE_PLACEHOLDER in content:\n                image_seqlen = image_grid_thw[num_image_tokens].prod() // merge_length if self.expand_mm_tokens else 1\n                content = content.replace(\n                    IMAGE_PLACEHOLDER,\n                    f\"{self.vision_bos_token}{self.image_token * image_seqlen}{self.vision_eos_token}\",\n                    1,\n                )\n                num_image_tokens += 1\n\n            if (\n                use_audio_in_video and len(audios) and len(videos)\n            ):  # if use the audio of video # deal video token and audio token together\n                if len(videos) != len(audios):\n                    raise ValueError(\n                        f\"Number of videos ({len(videos)}) must match number of audios ({len(audios)}) when using audio in video.\"\n                    )\n\n                while VIDEO_PLACEHOLDER in content:\n                    video_pos = content.find(VIDEO_PLACEHOLDER)\n                    audio_pos = content.find(AUDIO_PLACEHOLDER, video_pos)\n                    if audio_pos == -1 or audio_pos < video_pos:\n                        raise ValueError(\n                            f\"Each {VIDEO_PLACEHOLDER} must be followed by an {AUDIO_PLACEHOLDER} when using audio in video.\"\n                        )\n\n                    position_id_per_seconds: int = getattr(processor, \"position_id_per_seconds\", 25)\n                    audio_t_index = torch.arange(audio_lengths[num_audio_tokens])\n                    video_t_index = (\n                        torch.arange(video_grid_thw[num_video_tokens][0])\n                        .view(-1, 1, 1)\n                        .expand(\n                            -1,","sourceCodeStart":3002,"sourceCodeEnd":3038,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/mm_plugin.py#L3002-L3038","documentation":"In the Qwen2-Omni plugin, when use_audio_in_video is enabled and both audios and videos are present, every video must carry its own audio track: len(videos) must equal len(audios). The plugin then fuses each video's visual tokens with the corresponding audio's tokens using a shared time index.","triggerScenarios":"Template config with use_audio_in_video=true plus a dataset where a sample provides 2 <video> placeholders but 1 audio (or vice versa), including videos with no accompanying audio files.","commonSituations":"Enabling audio-in-video on a video-only dataset (audios column present for some rows only); misaligned video/audio columns after data conversion.","solutions":["Make the audios list length equal the videos list length — supply one audio track per video (even silent placeholders if the model expects them).","If videos have no meaningful audio, disable use_audio_in_video in the template config.","Filter rows where the counts differ before training."],"exampleFix":"### before\n{\"content\": \"<video>\", \"videos\": [\"a.mp4\", \"b.mp4\"], \"audios\": [\"a.wav\"]}\n### after\n{\"content\": \"<video>\", \"videos\": [\"a.mp4\", \"b.mp4\"], \"audios\": [\"a.wav\", \"b.wav\"]}","handlingStrategy":"validation","validationCode":"if use_audio_in_video and audios and videos:\n    assert len(videos) == len(audios), f'{len(videos)} videos vs {len(audios)} audios'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Generate the audios column alongside the videos column (extract WAV per MP4) in one prep script.","Do not enable use_audio_in_video on video-only data."],"tags":["multimodal","qwen2-omni","audio","video","dataset-validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}