{"record":{"id":"d2acedec6b631201","repo":"BoundaryML/baml","slug":"baml-internal-error-openai-responses-assistant-messages-must","errorCode":null,"errorMessage":"BAML internal error (openai-responses): assistant messages must be text; media not supported for assistant in Responses API","messagePattern":"BAML internal error \\(openai-responses\\): assistant messages must be text; media not supported for assistant in Responses API","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"engine/baml-runtime/src/internal/llm_client/primitive/openai/openai_client.rs","lineNumber":133,"sourceCode":"    role: &str,\n    allowed_metadata: &AllowedRoleMetadata,\n) -> Result<serde_json::Value> {\n    match part {\n        ChatMessagePart::Text(text) => {\n            let content_type = if role == \"assistant\" {\n                \"output_text\"\n            } else {\n                \"input_text\"\n            };\n            Ok(json!({\n                \"type\": content_type,\n                \"text\": text\n            }))\n        }\n        ChatMessagePart::Media(media) => {\n            // For assistant role, we only support text outputs in Responses API.\n            if role == \"assistant\" {\n                anyhow::bail!(\n                    \"BAML internal error (openai-responses): assistant messages must be text; media not supported for assistant in Responses API\"\n                );\n            }\n            match media.media_type {\n                baml_types::BamlMediaType::Image => {\n                    let image_url = match &media.content {\n                        baml_types::BamlMediaContent::Url(url_content) => url_content.url.clone(),\n                        baml_types::BamlMediaContent::Base64(b64_media) => {\n                            format!(\n                                \"data:{};base64,{}\",\n                                media.mime_type_as_ok()?,\n                                b64_media.base64\n                            )\n                        }\n                        baml_types::BamlMediaContent::File(_) => {\n                            anyhow::bail!(\n                                \"BAML internal error (openai-responses): image file should have been resolved, not processed directly.\"\n                            );","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/BoundaryML/baml/blob/bd85ce9dee1463ff04d27efd20531013a4ff46c1/engine/baml-runtime/src/internal/llm_client/primitive/openai/openai_client.rs#L115-L151","documentation":"When serializing chat messages for the OpenAI Responses API, BAML found a media (non-text) content part attached to an `assistant`-role message. The Responses API only permits text (`output_text`) content on assistant messages, so BAML bails with an internal-error marker instead of sending a request OpenAI would reject. This indicates chat history containing images/audio/PDF in an assistant turn reached the Responses API serializer.","triggerScenarios":"Calling an LLM function whose client uses the OpenAI Responses API while the chat history includes an assistant message containing media (e.g. prior response carried an image, or a multi-turn conversation had media inserted into the assistant role). Hit in `responses_content_part` when `role == \"assistant\"` and `ChatMessagePart::Media` is matched (openai_client.rs:133).","commonSituations":"Replaying conversation history where a previous model reply included generated media; constructing few-shot examples with image outputs under assistant turns; migrating a client from the standard Chat Completions API (which tolerates media in assistant messages differently) to the Responses API.","solutions":["Remove media parts from assistant-role messages in your prompt/history; keep assistant turns text-only.","Move media to user or system messages, which the Responses API accepts as input_image/input_audio/input_file.","If the media came from a stored previous response, strip non-text parts before re-sending the history.","If media-in-assistant is essential, use the standard OpenAI Chat Completions client instead of the Responses API strategy."],"exampleFix":"// before: assistant turn containing media in chat history\n{ role: \"assistant\", content: [media(image)] }\n\n// after: keep assistant text-only, move media to the user turn\n{ role: \"assistant\", content: [text(\"here is the analysis\")] }\n{ role: \"user\", content: [image(\"https://...\")] }","handlingStrategy":"validation","validationCode":"for msg in &history {\n    if msg.role == \"assistant\"\n        && msg.parts.iter().any(|p| matches!(p, ChatMessagePart::Media(_)))\n    {\n        panic!(\"assistant messages must be text-only for Responses API\");\n    }\n}","typeGuard":"fn is_text_only(msg: &RenderedChatMessage) -> bool {\n    msg.role != \"assistant\"\n        || msg.content.iter().all(|p| matches!(p, ChatMessagePart::Text(_)))\n}","tryCatchPattern":"match result {\n    Err(e) if e.to_string().contains(\"assistant messages must be text\") => {\n        // strip media from assistant turns and retry once\n    }\n    r => r?,\n}","preventionTips":["Keep media in user/system messages only; treat assistant turns as text-only in history builders.","Strip non-text parts when persisting and replaying assistant responses.","Audit few-shot examples for media in assistant roles before switching to the Responses API."],"tags":["openai","responses-api","media","assistant-message","rust"],"backgroundTag":"unsupported-operation","analyzedSha":"bd85ce9dee1463ff04d27efd20531013a4ff46c1","analyzedAt":"2026-09-12T03:38:25.718Z","contentChangedAt":"2026-09-12T03:38:25.718Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}