{"record":{"id":"f8857e434f74b29a","repo":"sgl-project/sglang","slug":"expected-f-h-w-c-uint8-video-got-shape-tupl","errorCode":null,"errorMessage":"Expected [F, H, W, C] uint8 video, got shape={tuple(video_uint8.shape)}","messagePattern":"Expected \\[F, H, W, C\\] uint8 video, got shape=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py","lineNumber":519,"sourceCode":"        device=device,\n        start_frame=target_start_frame,\n    )\n    return torch.cat([memory_coords, target_coords], dim=2)\n\n\n# --- Paired audio-video memory bank ---\n\n\n@dataclass\nclass MemoryEntry:\n    frame: Image.Image | list[Image.Image]\n    audio_latent: Optional[torch.Tensor] = None\n    metadata: dict[str, Any] = field(default_factory=dict)\n\n\ndef video_uint8_to_pil_frames(video_uint8: torch.Tensor) -> list[Image.Image]:\n    if video_uint8.ndim != 4:\n        raise ValueError(\n            f\"Expected [F, H, W, C] uint8 video, got shape={tuple(video_uint8.shape)}\"\n        )\n    if video_uint8.shape[-1] != 3:\n        raise ValueError(\n            f\"Expected RGB video with trailing channel dim 3, got shape={tuple(video_uint8.shape)}\"\n        )\n    video_uint8 = video_uint8.detach().cpu().contiguous()\n    return [Image.fromarray(frame.numpy()) for frame in video_uint8]\n\n\ndef normalize_audio_waveform_for_media(\n    audio_waveform: Optional[torch.Tensor],\n) -> Optional[torch.Tensor]:\n    if audio_waveform is None:\n        return None\n\n    waveform = torch.as_tensor(audio_waveform).detach().cpu().float()\n","sourceCodeStart":501,"sourceCodeEnd":537,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py#L501-L537","documentation":"video_uint8_to_pil_frames expects a 4-D [Frames, Height, Width, Channels] uint8 tensor because it iterates frames and does Image.fromarray on each one. Any other rank (e.g. a channel-first [C,F,H,W] or [F,H,W] video) is rejected immediately.","triggerScenarios":"Calling video_uint8_to_pil_frames (or the Joy-Echo memory stage's forward) with a CHW/NCFHW-format tensor from a torchvision/diffusers pipeline instead of NHWC, or with a single-frame [H,W,3] image.","commonSituations":"Mixing tensor conventions: many video pipelines output [C,T,H,W] or [T,C,H,W]; converting model output floats without adding the frame dim; passing a list/np.array instead of torch tensor of ndim 4.","solutions":["Permute/convert the tensor to [F,H,W,C] before calling: video = video.permute(0,2,3,1) for [F,C,H,W] input","If you have a single frame, unsqueeze the leading dim: frame.unsqueeze(0)","Check upstream decoder output format and standardize on NHWC uint8 for this API"],"exampleFix":"# before\nframes = video_uint8_to_pil_frames(video)  # video is [F,3,H,W]\n# after\nframes = video_uint8_to_pil_frames(video.permute(0, 2, 3, 1).contiguous())  # -> [F,H,W,3]","handlingStrategy":"type-guard","validationCode":"if video_uint8.ndim != 4 or video_uint8.shape[-1] != 3:\n    raise ValueError(f\"need [F,H,W,3] uint8, got {tuple(video_uint8.shape)}\")","typeGuard":"def is_fhwc_rgb_video(t: torch.Tensor) -> bool:\n    return isinstance(t, torch.Tensor) and t.ndim == 4 and t.shape[-1] == 3","tryCatchPattern":null,"preventionTips":["Standardize on NHWC uint8 at your pipeline boundary; permute once right after the decoder","Write a small assert helper for video tensors shared across stages"],"tags":["joy-echo","video","tensor-shape","nhwc","pil"],"backgroundTag":"invalid-tensor-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}