spring-projects/spring-ai · error · IllegalArgumentException

Unsupported media type: ${mimeType}

Error message

Unsupported media type: ${mimeType}

What it means

VertexAiMultimodalEmbeddingModel.doSingleDocumentPrediction handles IMAGE, VIDEO and TEXT media branches; any media item matching none of the supported MIME types falls into the final else branch, where a warning is logged and an IllegalArgumentException thrown. The Vertex multimodal embedding API only accepts image, video and text content, so other media cannot be embedded.

Source

Thrown at models/spring-ai-vertex-ai-embedding/src/main/java/org/springframework/ai/vertexai/embedding/multimodal/VertexAiMultimodalEmbeddingModel.java:210

				else if (logger.isWarnEnabled()) {
					logger.warn("Unsupported image mime type: " + media.getMimeType());
					throw new IllegalArgumentException("Unsupported image mime type: " + media.getMimeType());
				}
			}
			else if (media.getMimeType().isCompatibleWith(VIDEO_MIME_TYPE)) {
				instanceBuilder.video(VideoBuilder.of(media.getMimeType())
					.videoData(media.getData())
					.startOffsetSec(mergedOptions.getVideoStartOffsetSec())
					.endOffsetSec(mergedOptions.getVideoEndOffsetSec())
					.intervalSec(mergedOptions.getVideoIntervalSec())
					.build());
				documentMetadata.put(ModalityType.VIDEO,
						new DocumentMetadata(document.getId(), media.getMimeType(), media.getData()));
			}
			else {
				if (logger.isWarnEnabled()) {
					logger.warn("Unsupported media type: " + media.getMimeType());
				}
				throw new IllegalArgumentException("Unsupported media type: " + media.getMimeType());
			}
		}

		List<Value> instances = List.of(VertexAiEmbeddingUtils.valueOf(instanceBuilder.build()));

		PredictRequest.Builder predictRequestBuilder = PredictRequest.newBuilder()
			.setEndpoint(endpointName.toString())
			.setParameters(VertexAiEmbeddingUtils.jsonToValue(jsonHelper.toJson(Map.of())))
			.addAllInstances(instances);

		PredictResponse embeddingResponse = client.predict(predictRequestBuilder.build());

		int index = 0;
		List<Embedding> embeddingList = new ArrayList<>();
		for (Value prediction : embeddingResponse.getPredictionsList()) {
			if (prediction.getStructValue().containsFields("textEmbedding")) {
				Value textEmbedding = prediction.getStructValue().getFieldsOrThrow("textEmbedding");

View on GitHub (pinned to 98a7beda4f)

Solutions

1. Remove or convert unsupported media before embedding — the Vertex multimodal embedding API supports only image, video and text
2. Extract text (or frames) from unsupported formats yourself and pass them as text/image media
3. Validate the media MIME type in application code before constructing the Document

Example fix

// before
doc.getMedia().add(new Media(new MimeType("audio", "mpeg"), audioBytes));
// after
String transcript = transcribe(audioBytes);
Media textMedia = new Media(new MimeType("text", "plain"), transcript);

MimeType t = media.getMimeType();
boolean image = t != null && t.getType().equals("image");
boolean video = t != null && t.getType().equals("video");
if (!image && !video && !t.toString().startsWith("text/")) {
    throw new IllegalArgumentException("Unsupported media: " + t);
}

try {
    embeddingModel.embed(document);
} catch (IllegalArgumentException e) {
    // extract text/frames from unsupported media and retry
}

When it happens

Trigger: Embedding a Document containing Media whose MIME type is not compatible with IMAGE or VIDEO (and is not text) — e.g. application/pdf, audio/mp3, application/octet-stream.

Common situations: Passing audio files or PDFs to the multimodal embedding model; attaching binary blobs with generic MIME types; pipeline that forwards all attachments without filtering by modality.

Related errors

• Unsupported image mime type: ${mimeType}
• Media type String overrides the Document text content!
• Unable to detect the MIME type of '%s'. Please provide it ex
• Unsupported MimeType: %s
• Required properties for TitanEmbeddingBedrockApi are missing

AI-assisted analysis of spring-projects/spring-ai@98a7beda4f (2026-09-11). Data as JSON: /api/errors/e376a6acd3373652. Report an issue: GitHub.