{"record":{"id":"e376a6acd3373652","repo":"spring-projects/spring-ai","slug":"unsupported-media-type-mimetype","errorCode":null,"errorMessage":"Unsupported media type: ${mimeType}","messagePattern":"Unsupported media type: (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"models/spring-ai-vertex-ai-embedding/src/main/java/org/springframework/ai/vertexai/embedding/multimodal/VertexAiMultimodalEmbeddingModel.java","lineNumber":210,"sourceCode":"\t\t\t\telse if (logger.isWarnEnabled()) {\n\t\t\t\t\tlogger.warn(\"Unsupported image mime type: \" + media.getMimeType());\n\t\t\t\t\tthrow new IllegalArgumentException(\"Unsupported image mime type: \" + media.getMimeType());\n\t\t\t\t}\n\t\t\t}\n\t\t\telse if (media.getMimeType().isCompatibleWith(VIDEO_MIME_TYPE)) {\n\t\t\t\tinstanceBuilder.video(VideoBuilder.of(media.getMimeType())\n\t\t\t\t\t.videoData(media.getData())\n\t\t\t\t\t.startOffsetSec(mergedOptions.getVideoStartOffsetSec())\n\t\t\t\t\t.endOffsetSec(mergedOptions.getVideoEndOffsetSec())\n\t\t\t\t\t.intervalSec(mergedOptions.getVideoIntervalSec())\n\t\t\t\t\t.build());\n\t\t\t\tdocumentMetadata.put(ModalityType.VIDEO,\n\t\t\t\t\t\tnew DocumentMetadata(document.getId(), media.getMimeType(), media.getData()));\n\t\t\t}\n\t\t\telse {\n\t\t\t\tif (logger.isWarnEnabled()) {\n\t\t\t\t\tlogger.warn(\"Unsupported media type: \" + media.getMimeType());\n\t\t\t\t}\n\t\t\t\tthrow new IllegalArgumentException(\"Unsupported media type: \" + media.getMimeType());\n\t\t\t}\n\t\t}\n\n\t\tList<Value> instances = List.of(VertexAiEmbeddingUtils.valueOf(instanceBuilder.build()));\n\n\t\tPredictRequest.Builder predictRequestBuilder = PredictRequest.newBuilder()\n\t\t\t.setEndpoint(endpointName.toString())\n\t\t\t.setParameters(VertexAiEmbeddingUtils.jsonToValue(jsonHelper.toJson(Map.of())))\n\t\t\t.addAllInstances(instances);\n\n\t\tPredictResponse embeddingResponse = client.predict(predictRequestBuilder.build());\n\n\t\tint index = 0;\n\t\tList<Embedding> embeddingList = new ArrayList<>();\n\t\tfor (Value prediction : embeddingResponse.getPredictionsList()) {\n\t\t\tif (prediction.getStructValue().containsFields(\"textEmbedding\")) {\n\t\t\t\tValue textEmbedding = prediction.getStructValue().getFieldsOrThrow(\"textEmbedding\");","sourceCodeStart":192,"sourceCodeEnd":228,"githubUrl":"https://github.com/spring-projects/spring-ai/blob/98a7beda4f29d80a71c5837eb4053b03a93a46f7/models/spring-ai-vertex-ai-embedding/src/main/java/org/springframework/ai/vertexai/embedding/multimodal/VertexAiMultimodalEmbeddingModel.java#L192-L228","documentation":"VertexAiMultimodalEmbeddingModel.doSingleDocumentPrediction handles IMAGE, VIDEO and TEXT media branches; any media item matching none of the supported MIME types falls into the final else branch, where a warning is logged and an IllegalArgumentException thrown. The Vertex multimodal embedding API only accepts image, video and text content, so other media cannot be embedded.","triggerScenarios":"Embedding a Document containing Media whose MIME type is not compatible with IMAGE or VIDEO (and is not text) — e.g. application/pdf, audio/mp3, application/octet-stream.","commonSituations":"Passing audio files or PDFs to the multimodal embedding model; attaching binary blobs with generic MIME types; pipeline that forwards all attachments without filtering by modality.","solutions":["Remove or convert unsupported media before embedding — the Vertex multimodal embedding API supports only image, video and text","Extract text (or frames) from unsupported formats yourself and pass them as text/image media","Validate the media MIME type in application code before constructing the Document"],"exampleFix":"// before\ndoc.getMedia().add(new Media(new MimeType(\"audio\", \"mpeg\"), audioBytes));\n// after\nString transcript = transcribe(audioBytes);\nMedia textMedia = new Media(new MimeType(\"text\", \"plain\"), transcript);","handlingStrategy":"validation","validationCode":"MimeType t = media.getMimeType();\nboolean image = t != null && t.getType().equals(\"image\");\nboolean video = t != null && t.getType().equals(\"video\");\nif (!image && !video && !t.toString().startsWith(\"text/\")) {\n    throw new IllegalArgumentException(\"Unsupported media: \" + t);\n}","typeGuard":null,"tryCatchPattern":"try {\n    embeddingModel.embed(document);\n} catch (IllegalArgumentException e) {\n    // extract text/frames from unsupported media and retry\n}","preventionTips":["Allow only image/video/text media into Documents","Convert PDFs/audio to text or images first","Validate MIME types at ingestion time"],"tags":["vertex-ai","mime-type","multimodal","embedding"],"backgroundTag":"unsupported-media-type","analyzedSha":"98a7beda4f29d80a71c5837eb4053b03a93a46f7","analyzedAt":"2026-09-11T14:15:49.441Z","contentChangedAt":"2026-09-11T14:15:49.441Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}