{"record":{"id":"3db02316ba55555f","repo":"BoundaryML/baml","slug":"baml-internal-error-openai-responses-audio-must-be-base64","errorCode":null,"errorMessage":"BAML internal error (openai-responses): audio must be base64 encoded for Responses API","messagePattern":"BAML internal error \\(openai-responses\\): audio must be base64 encoded for Responses API","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"engine/baml-runtime/src/internal/llm_client/primitive/openai/openai_client.rs","lineNumber":173,"sourceCode":"                        \"type\": \"input_image\",\n                        \"detail\": \"auto\",\n                        \"image_url\": image_url\n                    }))\n                }\n                baml_types::BamlMediaType::Audio => match &media.content {\n                    baml_types::BamlMediaContent::Base64(b64_media) => {\n                        let mime_type = media.mime_type_as_ok()?;\n                        let format = mime_type.strip_prefix(\"audio/\").unwrap_or(&mime_type);\n                        Ok(json!({\n                            \"type\": \"input_audio\",\n                            \"input_audio\": {\n                                \"data\": b64_media.base64,\n                                \"format\": format\n                            }\n                        }))\n                    }\n                    _ => {\n                        anyhow::bail!(\n                            \"BAML internal error (openai-responses): audio must be base64 encoded for Responses API\"\n                        );\n                    }\n                },\n                baml_types::BamlMediaType::Pdf => match &media.content {\n                    baml_types::BamlMediaContent::Url(url_content) => Ok(json!({\n                        \"type\": \"input_file\",\n                        \"file_url\": url_content.url,\n                        \"filename\": \"document.pdf\"\n                    })),\n                    baml_types::BamlMediaContent::File(file_content) => {\n                        anyhow::bail!(\n                            \"BAML internal error (openai-responses): Local PDF files are not supported by OpenAI Responses API - use file_url for remote files or upload file and use file_id. File path: {:?}\",\n                            file_content.relpath\n                        );\n                    }\n                    baml_types::BamlMediaContent::Base64(b64_media) => Ok(json!({\n                        \"type\": \"input_file\",","sourceCodeStart":155,"sourceCodeEnd":191,"githubUrl":"https://github.com/BoundaryML/baml/blob/bd85ce9dee1463ff04d27efd20531013a4ff46c1/engine/baml-runtime/src/internal/llm_client/primitive/openai/openai_client.rs#L155-L191","documentation":"The OpenAI Responses API input_audio content part only accepts base64-encoded audio data with a format string. When an audio media part carried a URL or file reference instead of `BamlMediaContent::Base64`, BAML bails rather than sending an unsupported shape. Remote or local file audio must be converted to base64 before it can be sent through the Responses API.","triggerScenarios":"Sending audio to a function whose client uses the OpenAI Responses API with the audio provided as a URL (`BamlMediaContent::Url`) or as a local file (`BamlMediaContent::File`); only the Base64 variant is accepted (openai_client.rs:172-176, the `_` arm of the Audio match).","commonSituations":"Pointing `audio { url ... }` at a hosted recording while using a Responses-API model (e.g. gpt-4o-audio via Responses); passing a local audio file path; reusing prompt media config written for another provider that accepts URLs.","solutions":["Provide the audio as base64 (`audio { b64 ... }`) so the Responses API path can emit `input_audio`.","If the audio lives at a URL, download it and convert to base64 before passing it to the function.","If it is a local file, read and base64-encode it in your client code or use BAML's file-loading/media resolution to inline it.","Alternatively, switch the client to the standard OpenAI Chat Completions strategy if you need URL-based audio."],"exampleFix":"// before: URL audio on a Responses API client\naudio { url \"https://example.com/clip.mp3\" }\n\n// after: base64 audio\naudio { b64 \"SUQzBAAAAAAA...\" } // with mime_type audio/mp3","handlingStrategy":"validation","validationCode":"if let BamlMediaContent::Url(_) | BamlMediaContent::File(_) = audio.content {\n    // pre-fetch and base64-encode before sending\n    let bytes = fetch_bytes(&audio)?;\n    audio = to_base64_media(bytes, &audio.mime_type);\n}","typeGuard":"fn audio_is_base64(media: &BamlMedia) -> bool {\n    matches!(media.content, BamlMediaContent::Base64(_))\n}","tryCatchPattern":"match result {\n    Err(e) if e.to_string().contains(\"audio must be base64 encoded\") => {\n        // download/encode the audio to base64 and retry\n    }\n    r => r?,\n}","preventionTips":["Convert audio URLs/files to base64 before calling Responses-API functions.","Remember the Responses API input_audio only takes base64 data plus format.","Centralize media preparation so all audio is normalized to b64 for OpenAI clients."],"tags":["openai","responses-api","audio","base64","rust"],"backgroundTag":"unsupported-operation","analyzedSha":"bd85ce9dee1463ff04d27efd20531013a4ff46c1","analyzedAt":"2026-09-12T03:38:25.718Z","contentChangedAt":"2026-09-12T03:38:25.718Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}