{"record":{"id":"4775b0f8f8c271c9","repo":"BoundaryML/baml","slug":"video-input-is-only-supported-on-openai-s-realtime-api-v1","errorCode":null,"errorMessage":"Video input is only supported on OpenAI's Realtime API (/v1/realtime), not on chat completions. Consider extracting frames from the video as images instead. See: https://platform.openai.com/docs/guides/realtime","messagePattern":"Video input is only supported on OpenAI's Realtime API \\(/v1/realtime\\), not on chat completions\\. Consider extracting frames from the video as images instead\\. See: https://platform\\.openai\\.com/docs/guides/realtime","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"engine/baml-runtime/src/internal/llm_client/primitive/openai/openai_client.rs","lineNumber":894,"sourceCode":"                        content.insert(\n                            payload_key.into(),\n                            json!({\n                                \"filename\": \"document.pdf\",\n                                \"file_data\": format!(\"data:{};base64,{}\", media.mime_type_as_ok()?, b64_media.base64)\n                            }),\n                        );\n                    }\n                    BamlMediaContent::File(media_file) => {\n                        // For files, we need to resolve them to base64 first\n                        anyhow::bail!(\n                            \"BAML internal error (openai): Pdf file should have been resolved to base64 before this stage.\"\n                        );\n                    }\n                }\n            }\n            BamlMediaType::Video => {\n                // OpenAI video is only supported on the Realtime API (/v1/realtime), not on chat completions\n                anyhow::bail!(\n                    \"Video input is only supported on OpenAI's Realtime API (/v1/realtime), not on chat completions. \\\n                    Consider extracting frames from the video as images instead. \\\n                    See: https://platform.openai.com/docs/guides/realtime\"\n                );\n            }\n        }\n        Ok(content)\n    }\n\n    fn role_to_message(\n        &self,\n        content: &RenderedChatMessage,\n    ) -> Result<serde_json::Map<String, serde_json::Value>> {\n        let mut message = serde_json::Map::new();\n        message.insert(\"role\".into(), json!(content.role));\n\n        let strategy = self.get_provider_strategy();\n        let formatted_content =","sourceCodeStart":876,"sourceCodeEnd":912,"githubUrl":"https://github.com/BoundaryML/baml/blob/bd85ce9dee1463ff04d27efd20531013a4ff46c1/engine/baml-runtime/src/internal/llm_client/primitive/openai/openai_client.rs#L876-L912","documentation":"The OpenAI chat completions endpoint does not accept video input; BAML enforces this by bailing with a pointer to OpenAI's Realtime API, which is the only OpenAI surface that supports video. It suggests extracting video frames as images as a workaround.","triggerScenarios":"A ChatMessagePart with BamlMediaType::Video reaches to_media_message for the OpenAI chat completions converter (any video media in a prompt sent to OpenAI chat completions).","commonSituations":"Prompts that pass video files/URLs to gpt-4o-class models via chat completions, after seeing video support advertised for other providers or the Realtime API.","solutions":["Extract representative frames from the video and send them as image media instead","Route video to OpenAI's Realtime API (/v1/realtime) if live video is required","Use a provider that supports video input in chat requests (e.g. Gemini) in your BAML client config","Sample the video with ffmpeg and attach frames in the prompt"],"exampleFix":"// before (shell + baml)\nffmpeg -i video.mp4 frame_%03d.jpg\n// after (baml)\nimage { url \"./frame_001.jpg\" } image { url \"./frame_002.jpg\" }","handlingStrategy":"validation","validationCode":"// block video media for openai chat-completions clients\nif (media.type === \"video\" && provider === \"openai\" && api === \"chat-completions\") {\n  throw new Error(\"Video not supported on chat completions; extract frames or use Realtime API\");\n}","typeGuard":"function videoAllowed(provider: string, api: string): boolean {\n  return provider === \"openai\" && api === \"realtime\";\n}","tryCatchPattern":"// try video prompt, fall back to frame-extracted images\ntry {\n  return await fn.run({ video_url });\n} catch (e) {\n  if (String(e).includes(\"Video input is only supported\")) {\n    const frames = extractFrames(video_url);\n    return await fn.run({ frames });\n  }\n  throw e;\n}","preventionTips":["Extract video frames with ffmpeg before prompts when using OpenAI chat models","Configure a Gemini/other client for video-capable prompts","Document provider media capabilities in your prompt test suite"],"tags":["openai","video","chat-completions","unsupported-media"],"backgroundTag":"unsupported-operation","analyzedSha":"bd85ce9dee1463ff04d27efd20531013a4ff46c1","analyzedAt":"2026-09-12T03:38:25.718Z","contentChangedAt":"2026-09-12T03:38:25.718Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}