spring-projects/spring-ai · error · IllegalArgumentException

Unsupported media type

Error message

Unsupported media type: ${mimeType}

What it means

VertexAiMultimodalEmbeddingModel.doSingleDocumentPrediction handles IMAGE, VIDEO and TEXT media branches; any media item matching none of the supported MIME types falls into the final else branch, where a warning is logged and an IllegalArgumentException thrown. The Vertex multimodal embedding API only accepts image, video and text content, so other media cannot be embedded.

Solutions

1. Remove or convert unsupported media before embedding — the Vertex multimodal embedding API supports only image, video and text
2. Extract text (or frames) from unsupported formats yourself and pass them as text/image media
3. Validate the media MIME type in application code before constructing the Document

Example fix

// before
doc.getMedia().add(new Media(new MimeType("audio", "mpeg"), audioBytes));
// after
String transcript = transcribe(audioBytes);
Media textMedia = new Media(new MimeType("text", "plain"), transcript);

MimeType t = media.getMimeType();
boolean image = t != null && t.getType().equals("image");
boolean video = t != null && t.getType().equals("video");
if (!image && !video && !t.toString().startsWith("text/")) {
    throw new IllegalArgumentException("Unsupported media: " + t);
}

try {
    embeddingModel.embed(document);
} catch (IllegalArgumentException e) {
    // extract text/frames from unsupported media and retry
}

When it happens

Trigger: Embedding a Document containing Media whose MIME type is not compatible with IMAGE or VIDEO (and is not text) — e.g. application/pdf, audio/mp3, application/octet-stream.

Common situations: Passing audio files or PDFs to the multimodal embedding model; attaching binary blobs with generic MIME types; pipeline that forwards all attachments without filtering by modality.

Related errors

• Unsupported image mime type
• Media type String overrides the Document text content!
• Unable to detect the MIME type of
• Unsupported MimeType
• ai.onnxruntime.OrtException

AI-assisted analysis of spring-projects/spring-ai@98a7beda4f (2026-09-11). Data as JSON: /api/errors/e376a6acd3373652. Report an issue: GitHub.

Appendix: source

Thrown at models/spring-ai-vertex-ai-embedding/src/main/java/org/springframework/ai/vertexai/embedding/multimodal/VertexAiMultimodalEmbeddingModel.java:210

				else if (logger.isWarnEnabled()) {
					logger.warn("Unsupported image mime type: " + media.getMimeType());
					throw new IllegalArgumentException("Unsupported image mime type: " + media.getMimeType());
				}
			}
			else if (media.getMimeType().isCompatibleWith(VIDEO_MIME_TYPE)) {
				instanceBuilder.video(VideoBuilder.of(media.getMimeType())
					.videoData(media.getData())
					.startOffsetSec(mergedOptions.getVideoStartOffsetSec())
					.endOffsetSec(mergedOptions.getVideoEndOffsetSec())
					.intervalSec(mergedOptions.getVideoIntervalSec())
					.build());
				documentMetadata.put(ModalityType.VIDEO,
						new DocumentMetadata(document.getId(), media.getMimeType(), media.getData()));
			}
			else {
				if (logger.isWarnEnabled()) {
					logger.warn("Unsupported media type: " + media.getMimeType());
				}
				throw new IllegalArgumentException("Unsupported media type: " + media.getMimeType());
			}
		}

		List<Value> instances = List.of(VertexAiEmbeddingUtils.valueOf(instanceBuilder.build()));

		PredictRequest.Builder predictRequestBuilder = PredictRequest.newBuilder()
			.setEndpoint(endpointName.toString())
			.setParameters(VertexAiEmbeddingUtils.jsonToValue(jsonHelper.toJson(Map.of())))
			.addAllInstances(instances);

		PredictResponse embeddingResponse = client.predict(predictRequestBuilder.build());

		int index = 0;
		List<Embedding> embeddingList = new ArrayList<>();
		for (Value prediction : embeddingResponse.getPredictionsList()) {
			if (prediction.getStructValue().containsFields("textEmbedding")) {
				Value textEmbedding = prediction.getStructValue().getFieldsOrThrow("textEmbedding");

View on GitHub (pinned to 98a7beda4f)