{"record":{"id":"5dd3342f84a2e31a","repo":"sgl-project/sglang","slug":"paired-audio-memory-slot-requires-audio-latent","errorCode":null,"errorMessage":"paired audio memory slot requires audio_latent","messagePattern":"paired audio memory slot requires audio_latent","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py","lineNumber":700,"sourceCode":"        frames: list[Image.Image],\n        audio_latent: torch.Tensor,\n        *,\n        audio_window_size: int,\n        video_clip_num_frames: int,\n        audio_waveform: Optional[torch.Tensor] = None,\n        audio_sample_rate: int = 16000,\n        video_fps: float = 25.0,\n        audio_window_selection_mode: str = \"max_response\",\n        video_frame_selection_mode: str = \"center\",\n        audio_memory_mel_bins: int = 128,\n        audio_memory_mel_hop_length: int = 160,\n        audio_memory_n_fft: int = 1024,\n        audio_memory_downsample_factor: int = 4,\n        audio_memory_is_causal: bool = True,\n    ) -> dict[str, Any]:\n        audio_latent = self._prepare_audio_latent(audio_latent)\n        if audio_latent is None:\n            raise ValueError(\"paired audio memory slot requires audio_latent\")\n\n        selection_mode = str(audio_window_selection_mode).lower()\n        if audio_waveform is not None and selection_mode != \"center\":\n            try:\n                waveform = normalize_audio_waveform_for_media(audio_waveform)\n                mel = self._waveform_to_mel(\n                    waveform,\n                    sample_rate=audio_sample_rate,\n                    mel_bins=audio_memory_mel_bins,\n                    mel_hop_length=audio_memory_mel_hop_length,\n                    n_fft=audio_memory_n_fft,\n                )\n                pixel_window_size = latent_window_size_to_pixel_window_size(\n                    int(audio_window_size),\n                    downsample_factor=int(audio_memory_downsample_factor),\n                    is_causal=bool(audio_memory_is_causal),\n                )\n                _, window_start_indices, window_end_indices = (","sourceCodeStart":682,"sourceCodeEnd":718,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py#L682-L718","documentation":"save_memory_slot treats the slot as a paired audio-video memory slot when audio latents are expected: if _prepare_audio_latent returns None (audio_latent is None), the audio side of the memory cannot be built, so the call is rejected. You must either supply an audio latent or use the video-only slot API.","triggerScenarios":"Calling save_memory_slot with audio_latent=None while the slot type/args imply paired audio memory (e.g. audio_memory_n_fft etc. configured), or passing a latent that becomes None due to upstream conditional decoding that skipped audio.","commonSituations":"Videos without an audio track where the pipeline still routes to the paired audio-video memory path; upstream audio codec returning None on failure/empty audio and the value flowing through unchecked.","solutions":["Provide audio_latent (shape [B,T,C]) from your audio codec/VAE for the same clip","If the clip genuinely has no audio, use the video-only memory slot path (don't enable the paired audio memory config) instead of passing None","Fix the upstream decoder that silently returned None and add a guard/assert there"],"exampleFix":"# before\nsave_memory_slot(video=..., audio_latent=None)  # paired slot configured\n# after\nif audio_latent is None:\n    save_video_only_memory_slot(video=...)\nelse:\n    save_memory_slot(video=..., audio_latent=audio_latent)","handlingStrategy":"validation","validationCode":"if audio_latent is None:\n    raise ValueError(\"paired audio memory requires audio_latent; route to video-only slot instead\")","typeGuard":"def has_paired_audio(audio_latent: torch.Tensor | None) -> bool:\n    return audio_latent is not None and audio_latent.dim() == 3","tryCatchPattern":"try:\n    bank.save_memory_slot(video=v, audio_latent=al)\nexcept ValueError as e:\n    if \"requires audio_latent\" in str(e):\n        bank.save_memory_slot(video=v)  # video-only fallback\n    else:\n        raise","preventionTips":["Check audio presence before choosing paired vs video-only slot APIs","Guard the upstream audio decoder against silently returning None"],"tags":["joy-echo","audio","memory-slot","missing-argument"],"backgroundTag":"missing-required-parameter","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}