spring-projects/spring-ai · error · IllegalArgumentException

Unsupported image mime type:

Error message

Unsupported image mime type: 

What it means

Vertex AI multimodal embeddings only support a fixed set of image mime types (image/png, image/jpeg, etc.). If a Media in the document has an image-ish mime type not in the supported list, the model throws IllegalArgumentException after logging a warning. It is an eager fail-fast validation of unsupported input content.

Source

Thrown at models/spring-ai-vertex-ai-embedding/src/main/java/org/springframework/ai/vertexai/embedding/multimodal/VertexAiMultimodalEmbeddingModel.java:194

		Media media = document.getMedia();
		if (media != null) {
			if (media.getMimeType().isCompatibleWith(TEXT_MIME_TYPE)) {
				instanceBuilder.text(media.getData().toString());
				documentMetadata.put(ModalityType.TEXT,
						new DocumentMetadata(document.getId(), MimeTypeUtils.TEXT_PLAIN, media.getData()));
				if (logger.isWarnEnabled() && StringUtils.hasText(documentText)) {
					logger.warn("Media type String overrides the Document text content!");
				}
			}
			else if (media.getMimeType().isCompatibleWith(IMAGE_MIME_TYPE)) {
				if (SUPPORTED_IMAGE_MIME_SUB_TYPES.contains(media.getMimeType())) {
					instanceBuilder.image(ImageBuilder.of(media.getMimeType()).imageData(media.getData()).build());
					documentMetadata.put(ModalityType.IMAGE,
							new DocumentMetadata(document.getId(), media.getMimeType(), media.getData()));
				}
				else if (logger.isWarnEnabled()) {
					logger.warn("Unsupported image mime type: " + media.getMimeType());
					throw new IllegalArgumentException("Unsupported image mime type: " + media.getMimeType());
				}
			}
			else if (media.getMimeType().isCompatibleWith(VIDEO_MIME_TYPE)) {
				instanceBuilder.video(VideoBuilder.of(media.getMimeType())
					.videoData(media.getData())
					.startOffsetSec(mergedOptions.getVideoStartOffsetSec())
					.endOffsetSec(mergedOptions.getVideoEndOffsetSec())
					.intervalSec(mergedOptions.getVideoIntervalSec())
					.build());
				documentMetadata.put(ModalityType.VIDEO,
						new DocumentMetadata(document.getId(), media.getMimeType(), media.getData()));
			}
			else {
				if (logger.isWarnEnabled()) {
					logger.warn("Unsupported media type: " + media.getMimeType());
				}
				throw new IllegalArgumentException("Unsupported media type: " + media.getMimeType());
			}

View on GitHub (pinned to 98a7beda4f)

Solutions

1. Convert the image to a supported format (PNG or JPEG) before building the Document
2. Set an explicit MimeType on Media instead of relying on detection
3. Filter/validate media mime types against the supported image list before calling the model
4. Check the Vertex AI multimodalembedding docs for the current supported mime type list

Example fix

// before
Image image = new Image(ClassPathResource("photo.webp")); // image/webp -> throws
// after
BufferedImage img = ImageIO.read(new File("photo.webp"));
ImageIO.write(img, "png", new File("photo.png"));
Image image = new Image(new FileSystemResource("photo.png")); // image/png supported

static final Set<String> SUPPORTED = Set.of("image/png","image/jpeg","image/webp","image/heic","image/heif"); boolean ok = doc.getMedia().stream().allMatch(m -> SUPPORTED.contains(m.getMimeType().toString()));

static boolean isSupportedImage(MimeType mt) { return MediaType.parseMediaType("image/*").isCompatibleWith(mt) && Set.of("png","jpeg").contains(mt.getSubtype()); }

try { response = model.call(request); } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Unsupported image mime type")) { /* convert media and retry */ } throw e; }

When it happens

Trigger: Passing a Document containing Media whose MimeType is neither a supported image type nor video/text (e.g. image/webp, image/bmp, image/heic) via VertexAiMultimodalEmbeddingModel.call().

Common situations: Users embedding modern formats like WebP or HEIC that the multimodalembedding API does not accept; wrong mime type detected for a file; passing generic application/octet-stream data mislabeled as an image.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Unsupported media type:
• model cannot be null or empty
• java.io.IOException
• Title is only valid with task_type=RETRIEVAL_DOCUMENT
• Method must have exactly 1 parameter (List<McpSchema.Resourc

AI-assisted analysis of spring-projects/spring-ai@98a7beda4f (2026-09-11). Data as JSON: /api/errors/8f035f386cc1dace. Report an issue: GitHub.