{"record":{"id":"3d358ab6b8e178eb","repo":"langchain-ai/langchain","slug":"openai-messages-can-only-support-text-and-image-da","errorCode":null,"errorMessage":"OpenAI messages can only support text and image data. Received content block with media of type: {block['mime_type']}","messagePattern":"OpenAI messages can only support text and image data\\. Received content block with media of type: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"libs/core/langchain_core/messages/utils.py","lineNumber":1932,"sourceCode":"                        text = text[\"text\"]\n                    content.append({\"type\": \"text\", \"text\": text})\n                # VertexAI format\n                elif block.get(\"type\") == \"media\":\n                    if missing := [k for k in (\"mime_type\", \"data\") if k not in block]:\n                        err = (\n                            f\"Unrecognized content block at \"\n                            f\"messages[{i}].content[{j}] has 'type': \"\n                            f\"'media' but does not have key(s) {missing}. Full \"\n                            f\"content block:\\n\\n{block}\"\n                        )\n                        raise ValueError(err)\n                    if \"image\" not in block[\"mime_type\"]:\n                        err = (\n                            f\"OpenAI messages can only support text and image data.\"\n                            f\" Received content block with media of type:\"\n                            f\" {block['mime_type']}\"\n                        )\n                        raise ValueError(err)\n                    b64_image = _bytes_to_b64_str(block[\"data\"])\n                    content.append(\n                        {\n                            \"type\": \"image_url\",\n                            \"image_url\": {\n                                \"url\": (f\"data:{block['mime_type']};base64,{b64_image}\")\n                            },\n                        }\n                    )\n                elif (\n                    block.get(\"type\") in {\"thinking\", \"reasoning\"}\n                    or pass_through_unknown_blocks\n                ):\n                    content.append(block)\n                else:\n                    err = (\n                        f\"Unrecognized content block at \"\n                        f\"messages[{i}].content[{j}] does not match OpenAI, \"","sourceCodeStart":1914,"sourceCodeEnd":1950,"githubUrl":"https://github.com/langchain-ai/langchain/blob/e32fa9a52eab3b61ad7a45399bfde59b3e580fc4/libs/core/langchain_core/messages/utils.py#L1914-L1950","documentation":"After validating a VertexAI `media` block's keys, the converter checks that `mime_type` contains `\"image\"`. OpenAI's chat API accepts only text and image inputs, so video (`video/mp4`), audio (`audio/wav`), or application media cannot be represented and the conversion raises with the offending MIME type. This is a capability restriction of the target format, not a malformed block.","triggerScenarios":"`{\"type\": \"media\", \"mime_type\": \"video/mp4\", \"data\": ...}` or `{\"mime_type\": \"audio/ogg\", ...}` passed to `convert_to_openai_messages`; Gemini conversations containing inline video/audio parts forwarded to an OpenAI-compatible model.","commonSituations":"Routing the same multimodal history to both Gemini and OpenAI models; assuming `convert_to_openai_messages` silently drops unsupported modalities (it does not, unless `pass_through_unknown_blocks` applies — and it does not apply here since the block matched the `media` branch).","solutions":["Filter out non-image media blocks before conversion (see defense validationCode).","If audio input is required, use a provider that supports it natively (e.g. Gemini) rather than converting to OpenAI format.","For video, extract representative frames as images and replace the media block with `image_url` blocks.","Set expectations: text and images only when the destination is OpenAI."],"exampleFix":"// before\ncontent = [{\"type\": \"media\", \"mime_type\": \"video/mp4\", \"data\": raw}]\nconvert_to_openai_messages([HumanMessage(content=content)])\n\n// after\ncontent = [b for b in content if not (b.get(\"type\") == \"media\" and \"image\" not in b.get(\"mime_type\", \"\"))]\nconvert_to_openai_messages([HumanMessage(content=content)])","handlingStrategy":"validation","validationCode":"def is_openai_supported_media(b: dict) -> bool:\n    return not (b.get(\"type\") == \"media\" and \"image\" not in str(b.get(\"mime_type\", \"\")))\n\n# strip unsupported modalities before converting to OpenAI format\nblocks = [b for b in blocks if is_openai_supported_media(b)]","typeGuard":"def is_image_media(b: dict) -> bool:\n    return b.get(\"type\") == \"media\" and \"image\" in str(b.get(\"mime_type\", \"\"))","tryCatchPattern":"try:\n    oai = convert_to_openai_messages(messages)\nexcept ValueError as e:\n    if \"can only support text and image\" in str(e):\n        messages = filter_non_image_media(messages)  # your helper\n        oai = convert_to_openai_messages(messages)","preventionTips":["Route audio/video inputs to providers that support them natively instead of converting","Filter media by mime_type prefix at ingestion time","Document per-destination modality support in your routing layer"],"tags":["messages","multimodal","vertexai","openai","unsupported-format"],"backgroundTag":null,"analyzedSha":"e32fa9a52eab3b61ad7a45399bfde59b3e580fc4","analyzedAt":"2026-08-14T18:42:09.092Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}