langchain-ai/langchain · error · ValueError

Unrecognized content block at messages[{i}].content[{j}] has

Error message

Unrecognized content block at messages[{i}].content[{j}] has 'type': 'media' but does not have key(s) {missing}. Full content block:

{block}

What it means

VertexAI-format media blocks (`{"type": "media", ...}`) must include `mime_type` and `data`; the converter base64-encodes `data` and embeds it in a data-URI using `mime_type`. If either key is absent the message lists exactly which are missing (`missing` in the error text) and aborts `convert_to_openai_messages`.

Source

Thrown at libs/core/langchain_core/messages/utils.py:1925

                            f"'guard_content' but does not have a "
                            f"messages[{i}].content[{j}]['guard_content']['text'] "
                            f"key. Full content block:\n\n{block}"
                        )
                        raise ValueError(msg)
                    text = block["guard_content"]["text"]
                    if isinstance(text, dict):
                        text = text["text"]
                    content.append({"type": "text", "text": text})
                # VertexAI format
                elif block.get("type") == "media":
                    if missing := [k for k in ("mime_type", "data") if k not in block]:
                        err = (
                            f"Unrecognized content block at "
                            f"messages[{i}].content[{j}] has 'type': "
                            f"'media' but does not have key(s) {missing}. Full "
                            f"content block:\n\n{block}"
                        )
                        raise ValueError(err)
                    if "image" not in block["mime_type"]:
                        err = (
                            f"OpenAI messages can only support text and image data."
                            f" Received content block with media of type:"
                            f" {block['mime_type']}"
                        )
                        raise ValueError(err)
                    b64_image = _bytes_to_b64_str(block["data"])
                    content.append(
                        {
                            "type": "image_url",
                            "image_url": {
                                "url": (f"data:{block['mime_type']};base64,{b64_image}")
                            },
                        }
                    )
                elif (
                    block.get("type") in {"thinking", "reasoning"}

View on GitHub (pinned to e32fa9a52e)

Solutions

1. Include both keys: `{"type": "media", "mime_type": "image/png", "data": b"<raw bytes>"}`.
2. If blocks round-trip through JSON, base64-encode and decode `data` explicitly yourself, or switch to OpenAI `image_url` blocks which are JSON-safe.
3. Verify `mime_type` contains `image` — non-image media fails the next check (error 168) even with both keys present.

Example fix

// before
{"type": "media", "mime_type": "video/mp4", "data": raw}

// after (image only)
{"type": "media", "mime_type": "image/png", "data": raw}
// or JSON-safe OpenAI form:
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}}

def valid_media_block(b: dict) -> bool:
    return b.get("type") == "media" and "mime_type" in b and "data" in b

for b in blocks:
    if b.get("type") == "media" and not valid_media_block(b):
        raise ValueError(f"media block missing keys: {b}")

def is_complete_media_block(b: dict) -> bool:
    return (
        b.get("type") == "media"
        and isinstance(b.get("mime_type"), str)
        and ("data" in b)
    )

When it happens

Trigger: `{"type": "media", "mime_type": "image/png"}` without `data`; `{"type": "media", "data": b"..."}` without `mime_type`; VertexAI payloads where `data` is `None` and was dropped by a dict-compaction step.

Common situations: Converting Gemini/VertexAI multimodal histories to OpenAI format; storing media blocks in JSON (bytes are not JSON-serializable, so `data` gets stripped) and reloading them for conversion.

Related errors

• Unrecognized content block at messages[{i}].content[{j}] has
• Unrecognized content block at messages[{i}].content[{j}] has
• Unrecognized content block at messages[{i}].content[{j}] has
• Unrecognized content block at messages[{i}].content[{j}] has
• OpenAI messages can only support text and image data. Receiv

AI-assisted analysis of langchain-ai/langchain@e32fa9a52e (2026-08-14). Data as JSON: /api/errors/89fb1ec25bc4701b. Report an issue: GitHub.