sgl-project/sglang · error · ValueError

JoyEcho requires audio latents for denoising.

Error message

JoyEcho requires audio latents for denoising.

What it means

JoyEcho is an audio-conditioned video model; its denoising step requires ctx.audio_latents to be present. If the context was prepared without audio latents (no audio supplied, or the audio encode stage skipped), the step refuses to run rather than silently generating without conditioning.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/denoising.py:457

            ),
            {
                "memory_video_len": memory_video_len,
                "memory_audio_len": memory_audio_len,
                "late_layer_ratio": late_layer_ratio,
                "audio_replicated_for_sp": sp_on,
                "video_memory_prefix_len": memory_video_len if sp_on else 0,
            },
        )

    def _run_denoising_step(
        self,
        ctx: LTX2DenoisingContext,
        step: DenoisingStepState,
        batch: Req,
        server_args: ServerArgs,
    ) -> None:
        if ctx.audio_latents is None:
            raise ValueError("JoyEcho requires audio latents for denoising.")
        if ctx.audio_scheduler is None:
            raise ValueError("JoyEcho audio scheduler was not prepared.")

        sigmas = ctx.scheduler.sigmas
        if not isinstance(sigmas, torch.Tensor):
            raise ValueError("Expected scheduler.sigmas to be a tensor for JoyEcho.")

        sigma = sigmas[step.step_index].to(
            device=ctx.latents.device, dtype=torch.float32
        )
        sigma_next = sigmas[step.step_index + 1].to(
            device=ctx.latents.device, dtype=torch.float32
        )
        sigma_val = float(sigma.item())
        sigma_next_val = float(sigma_next.item())

        model_inputs = self._prepare_ltx2_model_inputs(
            ctx, step, batch, server_args, sigma

View on GitHub (pinned to 0132848349)

Solutions

1. Attach an audio track / TTS output to the request so audio latents are encoded
2. Verify the audio encoding stage ran and populated ctx.audio_latents before denoising
3. Use a non-JoyEcho (video-only) pipeline if no audio conditioning is intended

if ctx.audio_latents is None:
    raise ValueError('Request needs audio for JoyEcho')  # fail early, before loop start

def joyecho_ready(ctx) -> bool:
    return ctx.audio_latents is not None and ctx.audio_scheduler is not None

When it happens

Trigger: Invoking the JoyEcho denoising step when the DenoisingContext was built with audio_latents=None — typically because the request had no audio track or the audio encoder produced no latents.

Common situations: Sending video-only requests to a JoyEcho pipeline; an upstream audio encoder failing silently; misconfigured pipeline omitting the audio encode stage.

Related errors

• SP DMD renoise requires `batch.sp_audio_orig_num_frames`.
• JoyEcho audio scheduler was not prepared.
• audio_out_channels must be divisible by tp_size for TP-shard
• audio_num_frames must be provided for RoPE coordinate genera
• JoyEchoPipeline requires JoyEchoPipelineConfig, got {type(co

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5807ab9d96130eb5. Report an issue: GitHub.