{"record":{"id":"979fd656e6980a27","repo":"spring-projects/spring-ai","slug":"titan-embedding-does-not-support-batch-embedding","errorCode":null,"errorMessage":"Titan Embedding does not support batch embedding. Multiple API calls will be made.","messagePattern":"Titan Embedding does not support batch embedding\\. Multiple API calls will be made\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingModel.java","lineNumber":93,"sourceCode":"\t * @param inputType the input type to use.\n\t */\n\tpublic BedrockTitanEmbeddingModel withInputType(InputType inputType) {\n\t\tthis.inputType = inputType;\n\t\treturn this;\n\t}\n\n\t@Override\n\tpublic float[] embed(Document document) {\n\t\tString text = document.getText();\n\t\tAssert.state(text != null, \"Document text must not be null\");\n\t\treturn embed(text);\n\t}\n\n\t@Override\n\tpublic EmbeddingResponse call(EmbeddingRequest request) {\n\t\tAssert.notEmpty(request.getInstructions(), \"At least one text is required!\");\n\t\tif (request.getInstructions().size() != 1) {\n\t\t\tlogger.warn(\"Titan Embedding does not support batch embedding. Multiple API calls will be made.\");\n\t\t}\n\n\t\tList<Embedding> embeddings = new ArrayList<>();\n\t\tvar indexCounter = new AtomicInteger(0);\n\t\tint tokenUsage = 0;\n\n\t\tfor (String inputContent : request.getInstructions()) {\n\t\t\tvar apiRequest = createTitanEmbeddingRequest(inputContent, request.getOptions());\n\n\t\t\ttry {\n\t\t\t\tTitanEmbeddingResponse response = Observation\n\t\t\t\t\t.createNotStarted(\"bedrock.embedding\", this.observationRegistry)\n\t\t\t\t\t.lowCardinalityKeyValue(\"model\", \"titan\")\n\t\t\t\t\t.lowCardinalityKeyValue(\"input_type\", this.inputType.name().toLowerCase(Locale.ROOT))\n\t\t\t\t\t.highCardinalityKeyValue(\"input_length\", String.valueOf(inputContent.length()))\n\t\t\t\t\t.observe(() -> {\n\t\t\t\t\t\tTitanEmbeddingResponse r = this.embeddingApi.embedding(apiRequest);\n\t\t\t\t\t\tAssert.notNull(r, \"Embedding API returned null response\");","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/spring-projects/spring-ai/blob/98a7beda4f29d80a71c5837eb4053b03a93a46f7/models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingModel.java#L75-L111","documentation":"This is a warning (not an exception) logged by BedrockTitanEmbeddingModel.call() when an EmbeddingRequest contains more than one text instruction. Amazon Titan's embedding API accepts only one input per invocation, so the model silently fans the request out into multiple sequential/embedded API calls, one per instruction. Embeddings are still returned, correlated by their original index via indexCounter, but cost and latency scale with the number of inputs.","triggerScenarios":"Calling bedrockTitanEmbeddingModel.call(new EmbeddingRequest(List.of(\"a\", \"b\"), ...)) or embedding multiple documents in one call — any request where request.getInstructions().size() != 1.","commonSituations":"Batch-indexing documents for a vector store with EmbeddingClient.embed(List<String>) or VectorStore.add() where the store batches documents; migrating code written for OpenAI-style batch embedding endpoints to Titan; large RAG ingestion jobs that hit Bedrock throttling because of the per-input call fan-out.","solutions":["Accept the fan-out: batch requests yourself into groups sized to avoid Bedrock ThrottlingException, or tune the RetryTemplate.","If you always embed one text at a time, pass exactly one instruction to suppress the warning.","Switch to a Bedrock embedding model that supports batches (e.g. Cohere Embed) if multi-input per call matters.","Raise logging level for org.springframework.ai.bedrock.titan to ERROR only if the per-call cost is understood and acceptable."],"exampleFix":"// before\nEmbeddingResponse response = titanModel.call(new EmbeddingRequest(texts, new EmbeddingOptionsBuilder().build()));\n// after\ntexts.forEach(t -> responses.add(titanModel.call(new EmbeddingRequest(List.of(t), opts)))); // explicit per-input calls, easier rate-limit control","handlingStrategy":"validation","validationCode":"if (texts == null || texts.isEmpty()) throw new IllegalArgumentException(\"At least one text is required\");\n// knowledge: >1 text => one Bedrock API call per text; size batches to respect rate limits","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check instructions.size() before batching into a single call","Size your ingestion batches against Bedrock per-region quota","Configure a RetryTemplate with exponential backoff for ThrottlingException"],"tags":["aws-bedrock","embedding","batching","rate-limiting"],"backgroundTag":"unsupported-operation","analyzedSha":"98a7beda4f29d80a71c5837eb4053b03a93a46f7","analyzedAt":"2026-09-11T14:15:49.441Z","contentChangedAt":"2026-09-11T14:15:49.441Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}