vercel/ai · error · InvalidArgumentError

The OpenAI Realtime translation API only supports 24kHz 16-b

Error message

The OpenAI Realtime translation API only supports 24kHz 16-bit PCM input audio.

What it means

OpenAI's realtime translation endpoint only accepts 16-bit PCM audio at a 24kHz sample rate as input. validateOpenAISpeechTranslationInputAudioFormat checks the inputAudioFormat option and throws InvalidArgumentError if the format is not 'audio/pcm' or if an explicit sample rate other than 24000 is supplied.

Source

Thrown at packages/openai/src/speech-translation/openai-speech-translation-model.ts:357

          },
          noise_reduction: null,
        },
        output: {
          language: targetLanguage,
        },
      },
    },
  };
}

function validateOpenAISpeechTranslationInputAudioFormat(
  inputAudioFormat: SpeechTranslationModelV4StreamOptions['inputAudioFormat'],
) {
  if (
    inputAudioFormat.type !== 'audio/pcm' ||
    (inputAudioFormat.rate != null && inputAudioFormat.rate !== 24000)
  ) {
    throw new InvalidArgumentError({
      argument: 'inputAudioFormat',
      message:
        'The OpenAI Realtime translation API only supports 24kHz 16-bit PCM input audio.',
    });
  }
}

// The bearer token rides the `openai-insecure-api-key` subprotocol (native
// `WebSocket` cannot send headers) and the Authorization header is stripped:
// OpenAI rejects handshakes that send both auth channels.
function getOpenAIRealtimeConnection(
  headers: Record<string, string | undefined>,
): {
  protocols: string[];
  headers: Record<string, string | undefined>;
} {
  // last case-variant wins: combineHeaders keeps case-distinct keys and
  // spreads per-call headers after configuration headers

View on GitHub (pinned to 69428b1f8b)

Solutions

1. Convert input audio to raw 16-bit PCM at 24000 Hz before streaming
2. Pass inputAudioFormat: { type: 'audio/pcm', rate: 24000 } explicitly
3. Resample with a library (e.g. ffmpeg -ar 24000) in your ingestion pipeline
4. Omit rate only if your PCM is already known to be 24kHz

Example fix

// before
inputAudioFormat: { type: 'audio/pcm', rate: 16000 }
// after
inputAudioFormat: { type: 'audio/pcm', rate: 24000 }

const fmt = options.inputAudioFormat;
if (fmt && (fmt.type !== 'audio/pcm' || (fmt.rate != null && fmt.rate !== 24000))) {
  throw new Error('OpenAI translation requires 24kHz 16-bit PCM input audio');
}

function is24kPcm(fmt: { type: string; rate?: number }): boolean {
  return fmt.type === 'audio/pcm' && (fmt.rate === undefined || fmt.rate === 24000);
}

try {
  await translationModel.doStream(options);
} catch (e) {
  if (InvalidArgumentError.isInstance(e) && e.argument === 'inputAudioFormat') {
    // resample/transcode input to 24kHz PCM and retry
  } else throw e;
}

When it happens

Trigger: Calling doStream on the OpenAI speech translation model with inputAudioFormat set to a non-PCM type (e.g. audio/wav, audio/mp3) or with rate !== 24000 (e.g. 16000 or 44100).

Common situations: Feeding microphone audio captured at 16kHz or 48kHz; passing compressed file audio (wav containers, mp3) instead of raw PCM frames; reusing input formats valid for the transcription REST API with the realtime translation API.

Related errors

• targetLanguage is required for translation model '${this.mod
• ElevenLabs only supports audio/pcmu at 8000 Hz
• ElevenLabs realtime transcription supports audio/pcm at 8000
• The Gemini Live translation API only supports 16kHz 16-bit P
• non-streaming transcription with ${this.modelId}

AI-assisted analysis of vercel/ai@69428b1f8b (2026-08-30). Data as JSON: /api/errors/4419cc6167cf99b2. Report an issue: GitHub.