PrefectHQ/fastmcp · error · ValueError

ImageContent/AudioContent is only supported in user messages

Error message

ImageContent/AudioContent is only supported in user messages for OpenAI

What it means

OpenAI requires assistant messages to be text-only (plus tool_calls). When building an assistant message that includes tool_calls, if any content parts are non-text (image/audio detected by comparing part counts), the handler raises this ValueError.

Source

Thrown at fastmcp_slim/fastmcp/client/sampling/handlers/openai.py:259

                        tool_messages.append(
                            ChatCompletionToolMessageParam(
                                role="tool",
                                tool_call_id=item.tool_use_id,
                                content=content_text,
                            )
                        )
                    else:
                        raise ValueError(
                            f"Unsupported content type for OpenAI: {type(item).__name__}"
                        )

                # Add assistant message with tool calls if present
                # OpenAI requires: assistant (with tool_calls) -> tool messages
                if tool_calls or content_parts:
                    if tool_calls:
                        has_multimodal = len(content_parts) > len(text_parts)
                        if has_multimodal:
                            raise ValueError(
                                "ImageContent/AudioContent is only supported "
                                "in user messages for OpenAI"
                            )
                        text_str = "\n".join(text_parts) or None
                        openai_messages.append(
                            ChatCompletionAssistantMessageParam(
                                role="assistant",
                                content=text_str,
                                tool_calls=tool_calls,
                            )
                        )
                        # Add tool messages AFTER assistant message
                        openai_messages.extend(tool_messages)
                    elif content_parts:
                        if message.role == "user":
                            openai_messages.append(
                                ChatCompletionUserMessageParam(
                                    role="user",

View on GitHub (pinned to 1f02114297)

Solutions

Ensure assistant messages with tool_calls contain only TextContent.
Move images/audio into a following user message.
Strip non-text content from assistant messages before sampling.
Use a provider/handler that supports multimodal assistant messages (e.g. Anthropic or google-genai handler).

Example fix

// before
SamplingMessage(role='assistant', content=[TextContent(...), ImageContent(...)], tool_calls=[...])
// after
SamplingMessage(role='assistant', content=[TextContent(...)], tool_calls=[...])  # image moved to a user message

Defensive patterns

Strategy: validation

Validate before calling

for m in messages:
    if m.role == 'assistant' and getattr(m, 'tool_calls', None):
        if any(getattr(c, 'type', None) != 'text' for c in m.content):
            raise ValueError('Assistant tool_call messages must be text-only for OpenAI')

Type guard

def is_openai_safe_assistant_with_tools(m) -> bool:
    return not (m.role == 'assistant' and getattr(m, 'tool_calls', None)
                and any(getattr(c, 'type', None) != 'text' for c in m.content))

Try / catch

try:
    result = await client.sample(...)
except ValueError as e:
    if 'only supported in user messages for OpenAI' in str(e):
        messages = move_media_to_user_messages(messages)
        result = await client.sample(messages=messages, ...)
    else:
        raise

Prevention

Keep assistant turns with tool_calls text-only.
Replay histories from multimodal providers with media relocated to user turns.
Validate history structure before switching sampling providers.

When it happens

Trigger: A sampling message with role='assistant' that has tool_calls AND multimodal content (ImageContent/AudioContent) in its content list, converted during _convert_to_openai_messages.

Common situations: Replaying conversation histories where the assistant attached images alongside tool calls; MCP servers that echo rich assistant content; importing histories from providers (e.g. Claude) that allow multimodal assistant turns.

Related errors

AI-assisted analysis of PrefectHQ/fastmcp@1f02114297 (2026-08-29). Data as JSON: /api/errors/e490e56440d5847f. Report an issue: GitHub.