zylon-ai/private-gpt · error · ValueError

Failed to describe audio in the message.

Error message

Failed to describe audio in the message.

What it means

After process_audio_in_message ran with the audio-capable LLM, it returned an empty/falsy description. The call completed without exception but produced no usable transcript/description, which the preprocessor treats as failure rather than emitting an empty 'we processed these audios' message.

Source

Thrown at private_gpt/components/chat/processors/chat_history/multimodality/audio_preprocessor.py:70

    audio_blocks = extract_audio_blocks(message)
    if not audio_blocks:
        yield AudioProcessingResponse(message=message)
        return

    if audio_multimodal_llm is None:
        raise ValueError("Audio blocks found but no audio-capable LLM provided.")

    event = MultimodalProcessingStatus(status="processing", type="audio")
    yield AudioProcessingResponse(processing_status=event)

    try:
        audio_description = await process_audio_in_message(
            audio_multimodal_llm, message, user_query=message.content, **kwargs
        )

        if not audio_description:
            raise ValueError("Failed to describe audio in the message.")

        event = event.model_copy(
            update={
                "status": "completed",
                "content": audio_description,
            }
        )
        yield AudioProcessingResponse(processing_status=event)
        final_message = (
            "The user has included audios in their message. "
            "We have processed these audios and obtained the following descriptions:\n"
            f"{audio_description}"
        )

    except Errors.RequestTooLarge as e:
        event = event.model_copy(
            update={
                "status": "failed",

View on GitHub (pinned to 4a030776a3)

Solutions

1. Retry the request — transient empty completions often succeed on retry
2. Log/inspect the raw audio LLM response to find why the description is empty
3. Validate or transcode audio (format, duration, non-silence) before sending
4. If empties are expected, add a fallback description ('audio could not be described') in a wrapper

Example fix

# before
audio_description = await process_audio_in_message(llm, message, user_query=...)

# after
audio_description = await process_audio_in_message(llm, message, user_query=...)
if not audio_description:
    audio_description = "(audio content could not be transcribed)"  # or retry once

def is_describable_audio(message: ChatMessage) -> bool:
    return all(b_has_valid_audio(b) for b in extract_audio_blocks(message))  # non-empty, decodable payload

try:
    async for resp in audio_preprocessor.run(message, ...):
        ...
except ValueError as e:
    if "Failed to describe audio" in str(e):
        async for resp in audio_preprocessor.run(message, ...):  # one retry for transient empties
            ...
    else:
        raise

When it happens

Trigger: Calling the audio preprocessor with a message containing audio blocks where the audio LLM returns an empty string (empty completion, content filter, or unparsable response format).

Common situations: Audio model returning empty content on silence/undecodable audio; response parsing that extracts the wrong field; flaky inference server returning 200 with empty body.

Related errors

• Audio blocks found but no audio-capable LLM provided.
• Failed to describe images in the message.
• LLM does not support structured chat.
• No items returned from astream_structured_predict
• No items returned from astream_structured_chat

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/53a4ce1783aa0e03. Report an issue: GitHub.