HeyPuter/puter · error · HttpError

upstream_bad_request

upstream_bad_request

Error message

Gemini TTS did not return audio data

What it means

After Gemini returns from generateContent with responseModalities ['AUDIO'], GeminiTTSProvider extracts candidates[0].content.parts[0].inlineData.data. If that path is missing (no audio payload), it throws HTTP 400 (legacyCode upstream_bad_request) with fields.provider='gemini'. Commonly the model returned text or nothing instead of audio — e.g. the input tripped a safety filter, the model refused, or the response shape was unexpected.

Source

Thrown at src/backend/drivers/ai-tts/providers/gemini/GeminiTTSProvider.ts:253

        // status and caused 4xx validation errors to page.
        // eslint-disable-next-line @typescript-eslint/no-explicit-any
        const response: any = await this.#client.models.generateContent({
            model,
            contents: [{ parts: [{ text: inputText }] }],
            config: {
                responseModalities: ['AUDIO'],
                speechConfig: {
                    voiceConfig: {
                        prebuiltVoiceConfig: { voiceName: voice },
                    },
                },
            },
        });

        // Extract audio data from response
        const part = response?.candidates?.[0]?.content?.parts?.[0];
        if (!part?.inlineData?.data) {
            throw new HttpError(400, 'Gemini TTS did not return audio data', {
                legacyCode: 'upstream_bad_request',
                fields: { provider: 'gemini' },
            });
        }

        const audioBase64: string = part.inlineData.data;
        const mimeType: string =
            part.inlineData.mimeType || 'audio/L16;rate=24000';

        // Convert base64 PCM to a WAV buffer for broad client compatibility
        const pcmBuffer = Buffer.from(audioBase64, 'base64');
        let outputBuffer: Buffer;
        let contentType: string;

        if (mimeType.startsWith('audio/L16') || mimeType === 'audio/pcm') {
            // Wrap raw PCM (16-bit LE, 24kHz, mono) in a WAV container
            outputBuffer = this.#wrapPcmInWav(pcmBuffer, 24000, 1, 16);
            contentType = 'audio/wav';

View on GitHub (pinned to 908ec23eda)

Solutions

  1. Rephrase or sanitize the text to avoid content Gemini's safety filters reject, then retry.
  2. Confirm the model is a '-tts' model and the voice is a valid prebuilt name.
  3. Try the default voice/model to isolate whether the voice or model caused the empty audio.
  4. Inspect the full Gemini response server-side (add temporary logging of response.candidates) to see finishReason or safetyRatings.
  5. Retry once — transient empty responses can occur under load.
Defensive patterns

Strategy: fallback

Validate before calling

// Reduce the chance of an empty-audio response: sanitize/shorten text and
// stick to supported voice/model combos.
function sanitizeForGeminiTTS(text) {
  if (typeof text !== 'string' || !text.trim()) return null;
  // drop content likely to trip safety filters before sending
  return text.trim().slice(0, 5000);
}
const clean = sanitizeForGeminiTTS(userText);
if (clean) await driver.synthesize({ text: clean, provider: 'gemini', voice: 'Kore' });

Try / catch

try {
  await driver.synthesize({ text, provider: 'gemini', voice });
} catch (e) {
  if (e?.fields?.legacyCode === 'upstream_bad_request' && /did not return audio/.test(e.message)) {
    // likely a safety filter / model refusal — rephrase, switch voice/model, or fall back
    await driver.synthesize({ text: rephrased, provider: 'gemini', voice: 'Kore', model: 'gemini-2.5-flash-preview-tts' });
  } else throw e;
}

Prevention

When it happens

Trigger: Sending text Gemini declines to vocalize (disallowed content triggering safety filters), an empty/whitespace result after framing, a model/voice combo that returns text instead of audio, or an upstream response-format change. The provider does not retry; it surfaces the empty-audio result as a 400.

Common situations: Content that hits Gemini safety settings; very short or symbolic text the model doesn't treat as speech; API version drift where inlineData is nested differently; rate/quota responses that still return 200 but with no audio.

Related errors

AI-assisted analysis of HeyPuter/puter@908ec23eda (2026-08-12). Data as JSON: /api/errors/29c52bccf438cd0d. Report an issue: GitHub.