{"record":{"id":"4419cc6167cf99b2","repo":"vercel/ai","slug":"the-openai-realtime-translation-api-only-supports","errorCode":null,"errorMessage":"The OpenAI Realtime translation API only supports 24kHz 16-bit PCM input audio.","messagePattern":"The OpenAI Realtime translation API only supports 24kHz 16-bit PCM input audio\\.","errorType":"exception","errorClass":"InvalidArgumentError","httpStatus":null,"severity":"error","filePath":"packages/openai/src/speech-translation/openai-speech-translation-model.ts","lineNumber":357,"sourceCode":"          },\n          noise_reduction: null,\n        },\n        output: {\n          language: targetLanguage,\n        },\n      },\n    },\n  };\n}\n\nfunction validateOpenAISpeechTranslationInputAudioFormat(\n  inputAudioFormat: SpeechTranslationModelV4StreamOptions['inputAudioFormat'],\n) {\n  if (\n    inputAudioFormat.type !== 'audio/pcm' ||\n    (inputAudioFormat.rate != null && inputAudioFormat.rate !== 24000)\n  ) {\n    throw new InvalidArgumentError({\n      argument: 'inputAudioFormat',\n      message:\n        'The OpenAI Realtime translation API only supports 24kHz 16-bit PCM input audio.',\n    });\n  }\n}\n\n// The bearer token rides the `openai-insecure-api-key` subprotocol (native\n// `WebSocket` cannot send headers) and the Authorization header is stripped:\n// OpenAI rejects handshakes that send both auth channels.\nfunction getOpenAIRealtimeConnection(\n  headers: Record<string, string | undefined>,\n): {\n  protocols: string[];\n  headers: Record<string, string | undefined>;\n} {\n  // last case-variant wins: combineHeaders keeps case-distinct keys and\n  // spreads per-call headers after configuration headers","sourceCodeStart":339,"sourceCodeEnd":375,"githubUrl":"https://github.com/vercel/ai/blob/69428b1f8b037e4d118fb4853428d5c4e620493c/packages/openai/src/speech-translation/openai-speech-translation-model.ts#L339-L375","documentation":"OpenAI's realtime translation endpoint only accepts 16-bit PCM audio at a 24kHz sample rate as input. validateOpenAISpeechTranslationInputAudioFormat checks the inputAudioFormat option and throws InvalidArgumentError if the format is not 'audio/pcm' or if an explicit sample rate other than 24000 is supplied.","triggerScenarios":"Calling doStream on the OpenAI speech translation model with inputAudioFormat set to a non-PCM type (e.g. audio/wav, audio/mp3) or with rate !== 24000 (e.g. 16000 or 44100).","commonSituations":"Feeding microphone audio captured at 16kHz or 48kHz; passing compressed file audio (wav containers, mp3) instead of raw PCM frames; reusing input formats valid for the transcription REST API with the realtime translation API.","solutions":["Convert input audio to raw 16-bit PCM at 24000 Hz before streaming","Pass inputAudioFormat: { type: 'audio/pcm', rate: 24000 } explicitly","Resample with a library (e.g. ffmpeg -ar 24000) in your ingestion pipeline","Omit rate only if your PCM is already known to be 24kHz"],"exampleFix":"// before\ninputAudioFormat: { type: 'audio/pcm', rate: 16000 }\n// after\ninputAudioFormat: { type: 'audio/pcm', rate: 24000 }","handlingStrategy":"validation","validationCode":"const fmt = options.inputAudioFormat;\nif (fmt && (fmt.type !== 'audio/pcm' || (fmt.rate != null && fmt.rate !== 24000))) {\n  throw new Error('OpenAI translation requires 24kHz 16-bit PCM input audio');\n}","typeGuard":"function is24kPcm(fmt: { type: string; rate?: number }): boolean {\n  return fmt.type === 'audio/pcm' && (fmt.rate === undefined || fmt.rate === 24000);\n}","tryCatchPattern":"try {\n  await translationModel.doStream(options);\n} catch (e) {\n  if (InvalidArgumentError.isInstance(e) && e.argument === 'inputAudioFormat') {\n    // resample/transcode input to 24kHz PCM and retry\n  } else throw e;\n}","preventionTips":["Standardize your audio ingestion pipeline on 24kHz 16-bit mono PCM for realtime APIs","Resample microphone capture (often 44.1/48kHz) before streaming","Never pass compressed formats (wav/mp3) as inputAudioFormat to realtime translation"],"tags":["openai","speech-translation","audio-format","realtime","invalid-argument"],"backgroundTag":"unsupported-audio-format","analyzedSha":"69428b1f8b037e4d118fb4853428d5c4e620493c","analyzedAt":"2026-08-30T12:32:21.016Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}