{"record":{"id":"d92a518da44e8cba","repo":"sgl-project/sglang","slug":"video-must-be-a-tuple-of-video-tensor-timestamps","errorCode":null,"errorMessage":"video must be a tuple of (video_tensor, timestamps), but got {type(video)}. Video download and decoding should be done by sglang load_video before calling process_video.","messagePattern":"video must be a tuple of \\(video_tensor, timestamps\\), but got (.+?)\\. Video download and decoding should be done by sglang load_video before calling process_video\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/processors/mimo_v2.py","lineNumber":735,"sourceCode":"                if len(left_indices) > 0:\n                    selected_frame_indices = left_indices[-1:].clone()\n                else:\n                    raise ValueError(\n                        f\"No frames before start_time {start_time} in all_timestamps {all_timestamps.tolist()}\"\n                    )\n            else:\n                selected_frame_indices = candidate_indices\n\n            assert (\n                len(selected_frame_indices) > 0\n            ), f\"No frames selected for segment {start_time} - {end_time} in all_timestamps {all_timestamps.tolist()}\"\n            return selected_frame_indices\n\n        kwargs = self.prepare_video_kwargs(video_input)\n        video = video_input.video\n\n        if not isinstance(video, tuple):\n            raise ValueError(\n                f\"video must be a tuple of (video_tensor, timestamps), but got {type(video)}. \"\n                \"Video download and decoding should be done by sglang load_video before calling process_video.\"\n            )\n\n        video_tensor, timestamps_sampled = video\n        if len(timestamps_sampled) < 2:\n            logger.info(\n                \"[Warning] Less than two frames are sampled, using default fps (1 fps)\"\n            )\n            fps_sampled = 1\n        else:\n            fps_sampled = 1 / (timestamps_sampled[1] - timestamps_sampled[0])\n        num_frames_sampled = video_tensor.shape[0]\n\n        start_time = (\n            video_input.start_time\n            if video_input.start_time is not None\n            else timestamps_sampled[0]","sourceCodeStart":717,"sourceCodeEnd":753,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/processors/mimo_v2.py#L717-L753","documentation":"Raised by process_video when video_input.video is not a tuple of (video_tensor, timestamps). The MiMo-V2 pipeline expects decoding to have already been performed by sglang's load_video; passing raw bytes, a path, or a decord/HF video object directly violates that contract.","triggerScenarios":"Calling process_video with a VideoInput whose .video field holds undecoded data (path string, bytes, URL, or a HF video object) instead of the (tensor, timestamps) tuple produced by load_video.","commonSituations":"Bypassing the standard mm_data loading path in custom integrations; upgrading sglang versions where the internal video representation changed from raw to (tensor, timestamps); test code constructing VideoInput by hand.","solutions":["Run the video through sglang's load_video (the _preprocess_video_sync path) before process_video so .video is the (video_tensor, timestamps) tuple","If constructing VideoInput manually, decode first and set video=(frames_tensor, timestamps_tensor)","Align with the sglang version's expected internal representation after upgrades"],"exampleFix":"# before\nvi = VideoInput(video=b'raw mp4 bytes'); proc.process_video(vi)  # ValueError\n# after\nframes, ts = load_video('clip.mp4', **sampling_kwargs)  # via sglang loader\nvi = VideoInput(video=(frames, ts))\nout = proc.process_video(vi)","handlingStrategy":"type-guard","validationCode":"assert isinstance(video_input.video, tuple) and len(video_input.video) == 2, \\\n    'decode via sglang load_video first; .video must be (tensor, timestamps)'","typeGuard":"def is_decoded_video(v) -> bool:\n    return (isinstance(v, tuple) and len(v) == 2\n            and torch.is_tensor(v[0]) and torch.is_tensor(v[1]))","tryCatchPattern":"try:\n    out = proc.process_video(video_input)\nexcept ValueError as e:\n    if 'video must be a tuple' in str(e):\n        video_input.video = load_video(source, **sampling_kwargs)  # decode then retry\n        out = proc.process_video(video_input)\n    else:\n        raise","preventionTips":["Never construct VideoInput with raw bytes/paths on the process_video path","Always run load_video/_preprocess_video_sync first","After sglang upgrades, re-check the expected internal video representation"],"tags":["video","contract","type-error","decoding"],"backgroundTag":"wrong-argument-type","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}