sgl-project/sglang · error · ValueError

Expected audio_latent shape [B, T, C], got shape={tuple(audi

Error message

Expected audio_latent shape [B, T, C], got shape={tuple(audio_latent.shape)}

What it means

The Joy-Echo memory slot stores audio latents as 3-D [Batch, Time, Channels] tensors. _prepare_audio_latent rejects anything whose dim() != 3 because later concatenation (torch.cat along dim=1 in get_memory_audio) and window selection assume that layout.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py:581

    elif waveform.shape[0] > 2:
        waveform = waveform[:2]
    return waveform.contiguous()


class PairedAudioVideoMemoryBank:
    def __init__(self, max_size: int, num_fix_frames: int = 0) -> None:
        self.max_size = int(max_size)
        self.num_fix_frames = max(0, int(num_fix_frames))
        self.memory: list[MemoryEntry] = []

    @staticmethod
    def _prepare_audio_latent(
        audio_latent: Optional[torch.Tensor],
    ) -> Optional[torch.Tensor]:
        if audio_latent is None:
            return None
        if audio_latent.dim() != 3:
            raise ValueError(
                f"Expected audio_latent shape [B, T, C], got shape={tuple(audio_latent.shape)}"
            )
        return audio_latent.detach().cpu().contiguous()

    @staticmethod
    def _select_audio_window(
        audio_latent: torch.Tensor, window_size: int
    ) -> tuple[torch.Tensor, dict[str, Any]]:
        total_frames = int(audio_latent.shape[1])
        window_size = max(1, int(window_size))
        window_len = min(total_frames, window_size)
        window_start = max((total_frames - window_len) // 2, 0)
        window_end = window_start + window_len
        metadata = {
            "audio_window_start": int(window_start),
            "audio_window_end": int(window_end),
            "audio_window_length": int(window_len),
            "audio_total_frames": int(total_frames),

View on GitHub (pinned to 0132848349)

Solutions

1. Unsqueeze missing batch dim: audio_latent = audio_latent.unsqueeze(0) for [T, C] input
2. If the codec gives [B, C, T], transpose(1, 2) to [B, T, C] before saving
3. Verify B is 1 per slot (it must also match across slots for get_memory_audio)

Example fix

# before
save_memory_slot(..., audio_latent=latent)  # latent is [B, C, T]
# after
save_memory_slot(..., audio_latent=latent.transpose(1, 2).contiguous())  # [B, T, C]

if audio_latent is not None and audio_latent.dim() != 3:
    raise ValueError(f"audio_latent must be [B,T,C], got {tuple(audio_latent.shape)}; unsqueeze/transpose as needed")

def is_btc_audio_latent(t: torch.Tensor) -> bool:
    return isinstance(t, torch.Tensor) and t.dim() == 3

When it happens

Trigger: Calling save_memory_slot with audio_latent of shape [T, C] (missing batch dim), [B, C, T] (channels/time swapped), or a 4-D latent from a different codec format.

Common situations: Codec/VAE audio decoders returning [B, C, T] instead of [B, T, C]; forgetting unsqueeze(0) when saving a single clip's latent; format changes between model versions of the audio codec.

Related errors

• Expected batch size 1 for decoded audio, got shape={tuple(wa
• Expected decoded audio with 1, 2, or 3 dims, got shape={tupl
• paired audio memory slot requires audio_latent
• All memory audio latents must share batch and channel dimens
• Expected [F, H, W, C] uint8 video, got shape={tuple(video_ui

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3405b020190f5443. Report an issue: GitHub.