{"record":{"id":"cf513fe744fe6b9e","repo":"sgl-project/sglang","slug":"expected-rgb-video-with-trailing-channel-dim-3-go","errorCode":null,"errorMessage":"Expected RGB video with trailing channel dim 3, got shape={tuple(video_uint8.shape)}","messagePattern":"Expected RGB video with trailing channel dim 3, got shape=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py","lineNumber":523,"sourceCode":"\n\n# --- Paired audio-video memory bank ---\n\n\n@dataclass\nclass MemoryEntry:\n    frame: Image.Image | list[Image.Image]\n    audio_latent: Optional[torch.Tensor] = None\n    metadata: dict[str, Any] = field(default_factory=dict)\n\n\ndef video_uint8_to_pil_frames(video_uint8: torch.Tensor) -> list[Image.Image]:\n    if video_uint8.ndim != 4:\n        raise ValueError(\n            f\"Expected [F, H, W, C] uint8 video, got shape={tuple(video_uint8.shape)}\"\n        )\n    if video_uint8.shape[-1] != 3:\n        raise ValueError(\n            f\"Expected RGB video with trailing channel dim 3, got shape={tuple(video_uint8.shape)}\"\n        )\n    video_uint8 = video_uint8.detach().cpu().contiguous()\n    return [Image.fromarray(frame.numpy()) for frame in video_uint8]\n\n\ndef normalize_audio_waveform_for_media(\n    audio_waveform: Optional[torch.Tensor],\n) -> Optional[torch.Tensor]:\n    if audio_waveform is None:\n        return None\n\n    waveform = torch.as_tensor(audio_waveform).detach().cpu().float()\n\n    if waveform.ndim == 3:\n        if waveform.shape[0] != 1:\n            raise ValueError(\n                f\"Expected batch size 1 for decoded audio, got shape={tuple(waveform.shape)}\"","sourceCodeStart":505,"sourceCodeEnd":541,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py#L505-L541","documentation":"After the ndim==4 check, the trailing dimension must be exactly 3 (RGB) because Image.fromarray with the implied RGB mode requires a 3-channel last dim. RGBA, grayscale, or channel-first tensors that passed the 4-D check (e.g. [C,H,W,1]) trigger this.","triggerScenarios":"Passing an RGBA [F,H,W,4] video, a single-channel [F,H,W,1] grayscale video, or a mispermuted tensor whose last dim is not 3.","commonSituations":"Loading video with alpha channel from PNG sequences or screen captures; grayscale CCTV/medical footage fed without channel replication; tensors permuted so channels are not last.","solutions":["Drop alpha: video = video[..., :3] for RGBA input","Replicate grayscale to 3 channels: video = video.repeat(1, 1, 1, 3) (or use .convert('RGB') after decoding)","Re-check permutation so layout is genuinely [F,H,W,3]"],"exampleFix":"# before\nframes = video_uint8_to_pil_frames(rgba_video)  # [F,H,W,4]\n# after\nframes = video_uint8_to_pil_frames(rgba_video[..., :3].contiguous())  # [F,H,W,3]","handlingStrategy":"type-guard","validationCode":"if video_uint8.shape[-1] == 4:\n    video_uint8 = video_uint8[..., :3].contiguous()\nelif video_uint8.shape[-1] != 3:\n    raise ValueError(f\"need 3 trailing channels, got {video_uint8.shape[-1]}\")","typeGuard":"def has_rgb_trailing_dim(t: torch.Tensor) -> bool:\n    return t.shape[-1] == 3","tryCatchPattern":null,"preventionTips":["Convert RGBA/grayscale sources to RGB at load time (PIL .convert('RGB'))","Never assume decoder channel counts; log video.shape before the memory stage"],"tags":["joy-echo","video","rgb","channels","tensor-shape"],"backgroundTag":"invalid-tensor-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}