spring-projects/spring-ai · warning

Titan Embedding does not support batch embedding. Multiple A

Error message

Titan Embedding does not support batch embedding. Multiple API calls will be made.

What it means

This is a warning (not an exception) logged by BedrockTitanEmbeddingModel.call() when an EmbeddingRequest contains more than one text instruction. Amazon Titan's embedding API accepts only one input per invocation, so the model silently fans the request out into multiple sequential/embedded API calls, one per instruction. Embeddings are still returned, correlated by their original index via indexCounter, but cost and latency scale with the number of inputs.

Source

Thrown at models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/titan/BedrockTitanEmbeddingModel.java:93

	 * @param inputType the input type to use.
	 */
	public BedrockTitanEmbeddingModel withInputType(InputType inputType) {
		this.inputType = inputType;
		return this;
	}

	@Override
	public float[] embed(Document document) {
		String text = document.getText();
		Assert.state(text != null, "Document text must not be null");
		return embed(text);
	}

	@Override
	public EmbeddingResponse call(EmbeddingRequest request) {
		Assert.notEmpty(request.getInstructions(), "At least one text is required!");
		if (request.getInstructions().size() != 1) {
			logger.warn("Titan Embedding does not support batch embedding. Multiple API calls will be made.");
		}

		List<Embedding> embeddings = new ArrayList<>();
		var indexCounter = new AtomicInteger(0);
		int tokenUsage = 0;

		for (String inputContent : request.getInstructions()) {
			var apiRequest = createTitanEmbeddingRequest(inputContent, request.getOptions());

			try {
				TitanEmbeddingResponse response = Observation
					.createNotStarted("bedrock.embedding", this.observationRegistry)
					.lowCardinalityKeyValue("model", "titan")
					.lowCardinalityKeyValue("input_type", this.inputType.name().toLowerCase(Locale.ROOT))
					.highCardinalityKeyValue("input_length", String.valueOf(inputContent.length()))
					.observe(() -> {
						TitanEmbeddingResponse r = this.embeddingApi.embedding(apiRequest);
						Assert.notNull(r, "Embedding API returned null response");

View on GitHub (pinned to 98a7beda4f)

Solutions

1. Accept the fan-out: batch requests yourself into groups sized to avoid Bedrock ThrottlingException, or tune the RetryTemplate.
2. If you always embed one text at a time, pass exactly one instruction to suppress the warning.
3. Switch to a Bedrock embedding model that supports batches (e.g. Cohere Embed) if multi-input per call matters.
4. Raise logging level for org.springframework.ai.bedrock.titan to ERROR only if the per-call cost is understood and acceptable.

Example fix

// before
EmbeddingResponse response = titanModel.call(new EmbeddingRequest(texts, new EmbeddingOptionsBuilder().build()));
// after
texts.forEach(t -> responses.add(titanModel.call(new EmbeddingRequest(List.of(t), opts)))); // explicit per-input calls, easier rate-limit control

if (texts == null || texts.isEmpty()) throw new IllegalArgumentException("At least one text is required");
// knowledge: >1 text => one Bedrock API call per text; size batches to respect rate limits

When it happens

Trigger: Calling bedrockTitanEmbeddingModel.call(new EmbeddingRequest(List.of("a", "b"), ...)) or embedding multiple documents in one call — any request where request.getInstructions().size() != 1.

Common situations: Batch-indexing documents for a vector store with EmbeddingClient.embed(List<String>) or VectorStore.add() where the store batches documents; migrating code written for OpenAI-style batch embedding endpoints to Titan; large RAG ingestion jobs that hit Bedrock throttling because of the per-input call fan-out.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Tokens in a single document exceeds the maximum number of al
• Empty embedding vector returned for input at index + indexCo
• Required properties for TitanEmbeddingBedrockApi are missing
• InputType property for BedrockTitanEmbeddingModel is missing
• No embedding input is provided - all texts are null or empty

AI-assisted analysis of spring-projects/spring-ai@98a7beda4f (2026-09-11). Data as JSON: /api/errors/979fd656e6980a27. Report an issue: GitHub.