sgl-project/sglang · error · ValueError

Expected RGB video with trailing channel dim 3, got shape={t

Error message

Expected RGB video with trailing channel dim 3, got shape={tuple(video_uint8.shape)}

What it means

After the ndim==4 check, the trailing dimension must be exactly 3 (RGB) because Image.fromarray with the implied RGB mode requires a 3-channel last dim. RGBA, grayscale, or channel-first tensors that passed the 4-D check (e.g. [C,H,W,1]) trigger this.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py:523


# --- Paired audio-video memory bank ---


@dataclass
class MemoryEntry:
    frame: Image.Image | list[Image.Image]
    audio_latent: Optional[torch.Tensor] = None
    metadata: dict[str, Any] = field(default_factory=dict)


def video_uint8_to_pil_frames(video_uint8: torch.Tensor) -> list[Image.Image]:
    if video_uint8.ndim != 4:
        raise ValueError(
            f"Expected [F, H, W, C] uint8 video, got shape={tuple(video_uint8.shape)}"
        )
    if video_uint8.shape[-1] != 3:
        raise ValueError(
            f"Expected RGB video with trailing channel dim 3, got shape={tuple(video_uint8.shape)}"
        )
    video_uint8 = video_uint8.detach().cpu().contiguous()
    return [Image.fromarray(frame.numpy()) for frame in video_uint8]


def normalize_audio_waveform_for_media(
    audio_waveform: Optional[torch.Tensor],
) -> Optional[torch.Tensor]:
    if audio_waveform is None:
        return None

    waveform = torch.as_tensor(audio_waveform).detach().cpu().float()

    if waveform.ndim == 3:
        if waveform.shape[0] != 1:
            raise ValueError(
                f"Expected batch size 1 for decoded audio, got shape={tuple(waveform.shape)}"

View on GitHub (pinned to 0132848349)

Solutions

Drop alpha: video = video[..., :3] for RGBA input
Replicate grayscale to 3 channels: video = video.repeat(1, 1, 1, 3) (or use .convert('RGB') after decoding)
Re-check permutation so layout is genuinely [F,H,W,3]

Example fix

# before
frames = video_uint8_to_pil_frames(rgba_video)  # [F,H,W,4]
# after
frames = video_uint8_to_pil_frames(rgba_video[..., :3].contiguous())  # [F,H,W,3]

Defensive patterns

Strategy: type-guard

Validate before calling

if video_uint8.shape[-1] == 4:
    video_uint8 = video_uint8[..., :3].contiguous()
elif video_uint8.shape[-1] != 3:
    raise ValueError(f"need 3 trailing channels, got {video_uint8.shape[-1]}")

Type guard

def has_rgb_trailing_dim(t: torch.Tensor) -> bool:
    return t.shape[-1] == 3

Prevention

Convert RGBA/grayscale sources to RGB at load time (PIL .convert('RGB'))
Never assume decoder channel counts; log video.shape before the memory stage

When it happens

Trigger: Passing an RGBA [F,H,W,4] video, a single-channel [F,H,W,1] grayscale video, or a mispermuted tensor whose last dim is not 3.

Common situations: Loading video with alpha channel from PNG sequences or screen captures; grayscale CCTV/medical footage fed without channel replication; tensors permuted so channels are not last.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cf513fe744fe6b9e. Report an issue: GitHub.