spring-projects/spring-ai · error · IllegalArgumentException

Unsupported audio data type:

Error message

Unsupported audio data type: 

What it means

OpenAiChatModel.fromAudioData converts input-audio media data to a base64 string for the OpenAI API. It only accepts byte[]; any other object type (e.g. a String or InputStream) is rejected with this IllegalArgumentException naming the actual class received.

Source

Thrown at models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiChatModel.java:982

				return ChatCompletionToolChoiceOption.ofNamedToolChoice(named);
			case "auto":
				// There is a built-in “auto” option — but how to get it depends on SDK
				// version
				return ChatCompletionToolChoiceOption.ofAuto(ChatCompletionToolChoiceOption.Auto.AUTO);
			case "required":
				return ChatCompletionToolChoiceOption.ofAuto(ChatCompletionToolChoiceOption.Auto.REQUIRED);
			case "none":
				return ChatCompletionToolChoiceOption.ofAuto(ChatCompletionToolChoiceOption.Auto.NONE);
			default:
				throw new IllegalArgumentException("Unknown tool_choice type: " + type);
		}
	}

	private String fromAudioData(Object audioData) {
		if (audioData instanceof byte[] bytes) {
			return Base64.getEncoder().encodeToString(bytes);
		}
		throw new IllegalArgumentException("Unsupported audio data type: " + audioData.getClass().getSimpleName());
	}

	private String fromMediaData(org.springframework.util.MimeType mimeType, Object mediaContentData) {
		if (mediaContentData instanceof byte[] bytes) {
			// Assume the bytes are an image. So, convert the bytes to a base64 encoded
			// following the prefix pattern.
			return String.format("data:%s;base64,%s", mimeType.toString(), Base64.getEncoder().encodeToString(bytes));
		}
		else if (mediaContentData instanceof String text) {
			// Assume the text is a URLs or a base64 encoded image prefixed by the user.
			return text;
		}
		else {
			throw new IllegalArgumentException(
					"Unsupported media data type: " + mediaContentData.getClass().getSimpleName());
		}
	}

View on GitHub (pinned to 98a7beda4f)

Solutions

1. Read the audio into a byte[] (Files.readAllBytes or resource.getInputStream().readAllBytes()) before building the Media content.
2. If you already have a base64 String, decode it with Base64.getDecoder().decode(...) to byte[] first.
3. Verify which Message/Media constructor you used matches audio (byte[]) rather than URL-string media.

Example fix

// before
var media = new Media(MimeTypeUtils.parseMimeType("audio/wav"), base64String);
// after
byte[] bytes = Base64.getDecoder().decode(base64String);
var media = new Media(MimeTypeUtils.parseMimeType("audio/wav"), bytes);

if (!(audioData instanceof byte[])) {
    throw new IllegalArgumentException("Audio data must be byte[], got: " + (audioData == null ? "null" : audioData.getClass().getSimpleName()));
}

static boolean isSupportedAudioData(Object data) {
    return data instanceof byte[];
}

try {
    response = chatModel.call(prompt);
} catch (IllegalArgumentException e) {
    if (e.getMessage().startsWith("Unsupported audio data type")) {
        throw new InvalidMediaContentException("Convert audio content to byte[] before sending", e);
    }
    throw e;
}

When it happens

Trigger: Passing a Media/audio UserMessage content whose data is not byte[] — e.g. a base64 String or any other object — into OpenAiChatModel.call, reaching fromAudioData at OpenAiChatModel.java:982.

Common situations: Developers reading an audio file as a String, reusing image-content code that passes URL strings, or wiring a resource/loader object instead of raw bytes into an audio Media content.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Unsupported media data type:
• Unsupported media data type:
• Parameter must be of type List<McpSchema.Prompt>:
• Parameter must be of type List<McpSchema.Resource>:
• Parameter must be of type List<McpSchema.Tool>: " + method.g

AI-assisted analysis of spring-projects/spring-ai@98a7beda4f (2026-09-11). Data as JSON: /api/errors/0fc90eb22048002c. Report an issue: GitHub.