sgl-project/sglang · error · ValueError

paired audio memory slot requires audio_latent

Error message

paired audio memory slot requires audio_latent

What it means

save_memory_slot treats the slot as a paired audio-video memory slot when audio latents are expected: if _prepare_audio_latent returns None (audio_latent is None), the audio side of the memory cannot be built, so the call is rejected. You must either supply an audio latent or use the video-only slot API.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py:700

        frames: list[Image.Image],
        audio_latent: torch.Tensor,
        *,
        audio_window_size: int,
        video_clip_num_frames: int,
        audio_waveform: Optional[torch.Tensor] = None,
        audio_sample_rate: int = 16000,
        video_fps: float = 25.0,
        audio_window_selection_mode: str = "max_response",
        video_frame_selection_mode: str = "center",
        audio_memory_mel_bins: int = 128,
        audio_memory_mel_hop_length: int = 160,
        audio_memory_n_fft: int = 1024,
        audio_memory_downsample_factor: int = 4,
        audio_memory_is_causal: bool = True,
    ) -> dict[str, Any]:
        audio_latent = self._prepare_audio_latent(audio_latent)
        if audio_latent is None:
            raise ValueError("paired audio memory slot requires audio_latent")

        selection_mode = str(audio_window_selection_mode).lower()
        if audio_waveform is not None and selection_mode != "center":
            try:
                waveform = normalize_audio_waveform_for_media(audio_waveform)
                mel = self._waveform_to_mel(
                    waveform,
                    sample_rate=audio_sample_rate,
                    mel_bins=audio_memory_mel_bins,
                    mel_hop_length=audio_memory_mel_hop_length,
                    n_fft=audio_memory_n_fft,
                )
                pixel_window_size = latent_window_size_to_pixel_window_size(
                    int(audio_window_size),
                    downsample_factor=int(audio_memory_downsample_factor),
                    is_causal=bool(audio_memory_is_causal),
                )
                _, window_start_indices, window_end_indices = (

View on GitHub (pinned to 0132848349)

Solutions

1. Provide audio_latent (shape [B,T,C]) from your audio codec/VAE for the same clip
2. If the clip genuinely has no audio, use the video-only memory slot path (don't enable the paired audio memory config) instead of passing None
3. Fix the upstream decoder that silently returned None and add a guard/assert there

Example fix

# before
save_memory_slot(video=..., audio_latent=None)  # paired slot configured
# after
if audio_latent is None:
    save_video_only_memory_slot(video=...)
else:
    save_memory_slot(video=..., audio_latent=audio_latent)

if audio_latent is None:
    raise ValueError("paired audio memory requires audio_latent; route to video-only slot instead")

def has_paired_audio(audio_latent: torch.Tensor | None) -> bool:
    return audio_latent is not None and audio_latent.dim() == 3

try:
    bank.save_memory_slot(video=v, audio_latent=al)
except ValueError as e:
    if "requires audio_latent" in str(e):
        bank.save_memory_slot(video=v)  # video-only fallback
    else:
        raise

When it happens

Trigger: Calling save_memory_slot with audio_latent=None while the slot type/args imply paired audio memory (e.g. audio_memory_n_fft etc. configured), or passing a latent that becomes None due to upstream conditional decoding that skipped audio.

Common situations: Videos without an audio track where the pipeline still routes to the paired audio-video memory path; upstream audio codec returning None on failure/empty audio and the value flowing through unchecked.

Understand the failure class

Background: Missing required parameter errors: what 'X is required' and 'the required X param is missing' mean, and how to fix them — this error's family across 27 libraries.

Related errors

• Expected batch size 1 for decoded audio, got shape={tuple(wa
• Expected audio_latent shape [B, T, C], got shape={tuple(audi
• audio_num_frames must be provided for RoPE coordinate genera
• Expected decoded audio with 1, 2, or 3 dims, got shape={tupl
• All memory audio latents must share batch and channel dimens

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5dd3342f84a2e31a. Report an issue: GitHub.