sgl-project/sglang · error · ValueError

Whisper expects exactly 1 audio input, got {len}

Error message

Whisper expects exactly 1 audio input, got {len}

What it means

WhisperProcessor.process_mm_data_async handles strictly one audio input per request. audio_data must be truthy (else it returns None) and of length exactly 1; any batch of 2+ audio clips raises ValueError. Whisper's decoder is built around a single utterance per forward.

Source

Thrown at python/sglang/srt/multimodal/processors/whisper.py:178

                f"Language '{language}' is not in this Whisper model's vocabulary. "
                f"The '{language_token}' token may have been added in a later "
                f"Whisper version than the loaded checkpoint."
            )
        return token_id

    async def process_mm_data_async(
        self,
        image_data,
        audio_data,
        input_text,
        request_obj,
        **kwargs,
    ) -> Optional[Dict[str, Any]]:
        if not audio_data:
            return None

        if len(audio_data) != 1:
            raise ValueError(
                f"Whisper expects exactly 1 audio input, got {len(audio_data)}"
            )

        # Check if this is a fused auto-detect request (decoder prompt = [SOT] only,
        # structured generation handles the rest via regex constraint).
        detect_language = self._pop_sampling_param(request_obj, FUSED_AUTODETECT_FLAG)
        # timestamp_granularities is a transcription-level field; it must be
        # popped in both branches or it leaks into SamplingParams(**kwargs)
        # downstream and TypeErrors. In the fused branch the FSM regex was
        # already picked in build_fused_autodetect_params based on this value,
        # so we only need to keep it here to pick the timestamp_token_id for
        # the explicit-language branch.
        timestamp_granularities = self._pop_sampling_param(
            request_obj, "timestamp_granularities"
        )

        audios = [load_audio(audio) for audio in audio_data]

View on GitHub (pinned to 0132848349)

Solutions

1. Submit one audio clip per request and loop client-side
2. Concatenate clips into a single waveform before sending if they form one utterance
3. Fix request serialization so one clip is one element, not a nested list

Example fix

# before
resp = client.generate(prompt, audio_data=[clip1, clip2])
# after
for clip in [clip1, clip2]:
    resp = client.generate(prompt, audio_data=[clip])

if not audio_data or len(audio_data) != 1:
    raise ValueError('exactly one audio input required')

def is_single_audio(req) -> bool:
    a = req.get('audio_data') or req.get('audio')
    return isinstance(a, list) and len(a) == 1

When it happens

Trigger: Sending a multimodal request whose audio_data list contains 2 or more audio entries (multiple clips, chunked audio segments, or a mistranslated 'audio' array field).

Common situations: Splitting long recordings into chunks and submitting them together; frontend serializing a single audio as a list of segments; batching APIs reused for audio.

Related errors

• Language '{language}' not recognized. Use full name (e.g., '
• Language '{language}' is not in this Whisper model's vocabul
• audio_out_channels must be divisible by tp_size for TP-shard
• audio_num_frames must be provided for RoPE coordinate genera
• sound generation was requested (sound_duration > 0) but the

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/43a44112f6ceafa9. Report an issue: GitHub.