{"record":{"id":"3405b020190f5443","repo":"sgl-project/sglang","slug":"expected-audio-latent-shape-b-t-c-got-shape","errorCode":null,"errorMessage":"Expected audio_latent shape [B, T, C], got shape={tuple(audio_latent.shape)}","messagePattern":"Expected audio_latent shape \\[B, T, C\\], got shape=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py","lineNumber":581,"sourceCode":"    elif waveform.shape[0] > 2:\n        waveform = waveform[:2]\n    return waveform.contiguous()\n\n\nclass PairedAudioVideoMemoryBank:\n    def __init__(self, max_size: int, num_fix_frames: int = 0) -> None:\n        self.max_size = int(max_size)\n        self.num_fix_frames = max(0, int(num_fix_frames))\n        self.memory: list[MemoryEntry] = []\n\n    @staticmethod\n    def _prepare_audio_latent(\n        audio_latent: Optional[torch.Tensor],\n    ) -> Optional[torch.Tensor]:\n        if audio_latent is None:\n            return None\n        if audio_latent.dim() != 3:\n            raise ValueError(\n                f\"Expected audio_latent shape [B, T, C], got shape={tuple(audio_latent.shape)}\"\n            )\n        return audio_latent.detach().cpu().contiguous()\n\n    @staticmethod\n    def _select_audio_window(\n        audio_latent: torch.Tensor, window_size: int\n    ) -> tuple[torch.Tensor, dict[str, Any]]:\n        total_frames = int(audio_latent.shape[1])\n        window_size = max(1, int(window_size))\n        window_len = min(total_frames, window_size)\n        window_start = max((total_frames - window_len) // 2, 0)\n        window_end = window_start + window_len\n        metadata = {\n            \"audio_window_start\": int(window_start),\n            \"audio_window_end\": int(window_end),\n            \"audio_window_length\": int(window_len),\n            \"audio_total_frames\": int(total_frames),","sourceCodeStart":563,"sourceCodeEnd":599,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py#L563-L599","documentation":"The Joy-Echo memory slot stores audio latents as 3-D [Batch, Time, Channels] tensors. _prepare_audio_latent rejects anything whose dim() != 3 because later concatenation (torch.cat along dim=1 in get_memory_audio) and window selection assume that layout.","triggerScenarios":"Calling save_memory_slot with audio_latent of shape [T, C] (missing batch dim), [B, C, T] (channels/time swapped), or a 4-D latent from a different codec format.","commonSituations":"Codec/VAE audio decoders returning [B, C, T] instead of [B, T, C]; forgetting unsqueeze(0) when saving a single clip's latent; format changes between model versions of the audio codec.","solutions":["Unsqueeze missing batch dim: audio_latent = audio_latent.unsqueeze(0) for [T, C] input","If the codec gives [B, C, T], transpose(1, 2) to [B, T, C] before saving","Verify B is 1 per slot (it must also match across slots for get_memory_audio)"],"exampleFix":"# before\nsave_memory_slot(..., audio_latent=latent)  # latent is [B, C, T]\n# after\nsave_memory_slot(..., audio_latent=latent.transpose(1, 2).contiguous())  # [B, T, C]","handlingStrategy":"type-guard","validationCode":"if audio_latent is not None and audio_latent.dim() != 3:\n    raise ValueError(f\"audio_latent must be [B,T,C], got {tuple(audio_latent.shape)}; unsqueeze/transpose as needed\")","typeGuard":"def is_btc_audio_latent(t: torch.Tensor) -> bool:\n    return isinstance(t, torch.Tensor) and t.dim() == 3","tryCatchPattern":null,"preventionTips":["Normalize codec output to [B,T,C] in one adapter function used by all save paths","Pin the codec version so latent rank/layout doesn't drift"],"tags":["joy-echo","audio","latent","tensor-shape","memory-slot"],"backgroundTag":"invalid-tensor-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}