sgl-project/sglang · error · ValueError

Expected [F, H, W, C] uint8 video, got shape={tuple(video_ui

Error message

Expected [F, H, W, C] uint8 video, got shape={tuple(video_uint8.shape)}

What it means

video_uint8_to_pil_frames expects a 4-D [Frames, Height, Width, Channels] uint8 tensor because it iterates frames and does Image.fromarray on each one. Any other rank (e.g. a channel-first [C,F,H,W] or [F,H,W] video) is rejected immediately.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py:519

        device=device,
        start_frame=target_start_frame,
    )
    return torch.cat([memory_coords, target_coords], dim=2)


# --- Paired audio-video memory bank ---


@dataclass
class MemoryEntry:
    frame: Image.Image | list[Image.Image]
    audio_latent: Optional[torch.Tensor] = None
    metadata: dict[str, Any] = field(default_factory=dict)


def video_uint8_to_pil_frames(video_uint8: torch.Tensor) -> list[Image.Image]:
    if video_uint8.ndim != 4:
        raise ValueError(
            f"Expected [F, H, W, C] uint8 video, got shape={tuple(video_uint8.shape)}"
        )
    if video_uint8.shape[-1] != 3:
        raise ValueError(
            f"Expected RGB video with trailing channel dim 3, got shape={tuple(video_uint8.shape)}"
        )
    video_uint8 = video_uint8.detach().cpu().contiguous()
    return [Image.fromarray(frame.numpy()) for frame in video_uint8]


def normalize_audio_waveform_for_media(
    audio_waveform: Optional[torch.Tensor],
) -> Optional[torch.Tensor]:
    if audio_waveform is None:
        return None

    waveform = torch.as_tensor(audio_waveform).detach().cpu().float()

View on GitHub (pinned to 0132848349)

Solutions

1. Permute/convert the tensor to [F,H,W,C] before calling: video = video.permute(0,2,3,1) for [F,C,H,W] input
2. If you have a single frame, unsqueeze the leading dim: frame.unsqueeze(0)
3. Check upstream decoder output format and standardize on NHWC uint8 for this API

Example fix

# before
frames = video_uint8_to_pil_frames(video)  # video is [F,3,H,W]
# after
frames = video_uint8_to_pil_frames(video.permute(0, 2, 3, 1).contiguous())  # -> [F,H,W,3]

if video_uint8.ndim != 4 or video_uint8.shape[-1] != 3:
    raise ValueError(f"need [F,H,W,3] uint8, got {tuple(video_uint8.shape)}")

def is_fhwc_rgb_video(t: torch.Tensor) -> bool:
    return isinstance(t, torch.Tensor) and t.ndim == 4 and t.shape[-1] == 3

When it happens

Trigger: Calling video_uint8_to_pil_frames (or the Joy-Echo memory stage's forward) with a CHW/NCFHW-format tensor from a torchvision/diffusers pipeline instead of NHWC, or with a single-frame [H,W,3] image.

Common situations: Mixing tensor conventions: many video pipelines output [C,T,H,W] or [T,C,H,W]; converting model output floats without adding the frame dim; passing a list/np.array instead of torch tensor of ndim 4.

Related errors

• Expected RGB video with trailing channel dim 3, got shape={t
• Unsupported dims: {self.dims}
• Invalid latent grid for memory RoPE: {latent_height=} {laten
• Expected decoded audio with 1, 2, or 3 dims, got shape={tupl
• Expected audio_latent shape [B, T, C], got shape={tuple(audi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f8857e434f74b29a. Report an issue: GitHub.