sgl-project/sglang · error · ValueError

Audio placeholder count does not match audio_data

Error message

Audio placeholder count does not match audio_data

What it means

Raised by _merge_video_media when an audio marker is present in the merged prompt but the audio_data iterator is exhausted. Audio placeholders and supplied audio_data must be one-to-one.

Source

Thrown at python/sglang/srt/multimodal/processors/dots_note_omni.py:342

            rendered.append(input_text[last : match.start()])
            marker = match.group(0)
            expanded_media = video_media.get(marker)
            if expanded_media is not None:
                modality, value = expanded_media
            else:
                modality = self.mm_tokens.get_modality_of_token(marker)
                if modality == Modality.IMAGE:
                    try:
                        value = next(native_images)
                    except StopIteration as exc:
                        raise ValueError(
                            "Image placeholder count does not match image_data"
                        ) from exc
                elif modality == Modality.AUDIO:
                    try:
                        value = next(native_audios)
                    except StopIteration as exc:
                        raise ValueError(
                            "Audio placeholder count does not match audio_data"
                        ) from exc
                else:
                    raise ValueError(f"Unsupported dots omni media marker: {marker}")

            if modality == Modality.IMAGE:
                ordered_images.append(value)
                rendered.append(
                    self.image_start_token + self.image_token + self.image_end_token
                )
            else:
                ordered_audios.append(value)
                rendered.append(
                    self.audio_start_token + self.audio_token + self.audio_end_token
                )
            last = match.end()

        rendered.append(input_text[last:])

View on GitHub (pinned to 0132848349)

Solutions

1. Provide one audio entry per audio marker, or drop audio markers from the prompt/template
2. Disable audio processing with video_config audio_cap=0 if no audio is being sent
3. Check for duplicate rendering when multiple videos are flattened into one prompt

Example fix

// before
video_config = {"audio_cap": 1.0}  # renders audio markers
audio_data = []
// after
video_config = {"audio_cap": 0}  # no audio markers expected
audio_data = []

audio_markers = count_audio_markers(prompt)  # e.g. regex <|sglang_dots_video_\d+_audio_\d+|>
assert audio_markers == len(audio_data or []), 'audio marker/audio_data mismatch'

try:
    await processor.process_mm_data_async(...)
except ValueError as e:
    if 'Audio placeholder count' in str(e):
        audio_data = audio_data[:audio_markers] or add markers; retry

When it happens

Trigger: Video content or the prompt contains more audio markers (e.g. <|sglang_dots_video_N_audio_M|>) than entries in the request's audio_data list.

Common situations: The chat template renders an audio tag per video while the client omits audio_data, or sends fewer audio clips than videos processed with audio enabled (audio_cap > 0).

Related errors

• audio_cap must be non-negative, got {audio_cap}
• audio_sr must be positive, got {audio_sr}
• Dots omni audio must be mono, got shape={tuple(waveform.shap
• Image placeholder count does not match image_data
• Video placeholder count does not match video_data: {len(vide

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/701657a9f54109d0. Report an issue: GitHub.