spring-projects/spring-ai · error · IllegalArgumentException

Tokens in a single document exceeds the maximum number of al

Error message

Tokens in a single document exceeds the maximum number of allowed input tokens

What it means

TokenCountBatchingStrategy.batch() splits documents into batches whose total estimated token count stays under maxInputTokenCount. If a single document alone exceeds that maximum, it can never fit in a valid batch, so the strategy throws IllegalArgumentException instead of producing a batch that the model would reject.

Source

Thrown at spring-ai-model/src/main/java/org/springframework/ai/embedding/TokenCountBatchingStrategy.java:148

		this.tokenCountEstimator = tokenCountEstimator;
		this.maxInputTokenCount = (int) Math.round(maxInputTokenCount * (1 - reservePercentage));
		this.contentFormatter = contentFormatter;
		this.metadataMode = metadataMode;
	}

	@Override
	public List<List<Document>> batch(List<Document> documents) {
		List<List<Document>> batches = new ArrayList<>();
		int currentSize = 0;
		List<Document> currentBatch = new ArrayList<>();

		// Do not collect the documents into a Map keyed by Document: equal documents
		// would collapse to a single entry and be silently dropped from the batches.
		for (Document document : documents) {
			int tokenCount = this.tokenCountEstimator
				.estimate(document.getFormattedContent(this.contentFormatter, this.metadataMode));
			if (tokenCount > this.maxInputTokenCount) {
				throw new IllegalArgumentException(
						"Tokens in a single document exceeds the maximum number of allowed input tokens");
			}
			currentSize += tokenCount;
			if (currentSize > this.maxInputTokenCount) {
				batches.add(currentBatch);
				currentBatch = new ArrayList<>();
				currentSize = tokenCount;
			}
			currentBatch.add(document);
		}
		if (!currentBatch.isEmpty()) {
			batches.add(currentBatch);
		}
		return batches;
	}

}

View on GitHub (pinned to 98a7beda4f)

Solutions

Split large documents into smaller chunks (e.g. TokenTextSplitter) before passing them to the embedding model
Raise TokenCountBatchingStrategy.maxInputTokenCount via its constructor/builder to match your embedding model's actual limit
Shorten document content or reduce metadata included in formattedContent (adjust ContentFormatter/MetadataMode) to lower the estimated token count

Example fix

// before
embeddingModel.embed(documents); // one doc exceeds max tokens
// after
List<Document> chunks = new TokenTextSplitter().apply(documents);
embeddingModel.embed(chunks);

Defensive patterns

Strategy: validation

Validate before calling

TokenCountBatchingStrategy strategy = new TokenCountBatchingStrategy();
for (Document doc : documents) {
    int tokens = strategy.getTokenCountEstimator()
        .estimate(doc.getFormattedContent());
    if (tokens > strategy.getMaxInputTokenCount()) {
        throw new IllegalStateException("Document too large: " + doc.getId() + " tokens=" + tokens);
    }
}

Try / catch

try {
    embeddingModel.embed(documents);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("exceeds the maximum number of allowed input tokens")) {
        List<Document> chunks = new TokenTextSplitter().apply(documents);
        embeddingModel.embed(chunks);
    } else {
        throw e;
    }
}

Prevention

Always chunk large documents with TokenTextSplitter before embedding
Set maxInputTokenCount to your embedding model's documented input limit with headroom
Keep an eye on formattedContent size — metadata included via ContentFormatter counts toward tokens

When it happens

Trigger: Calling batch(docs) (directly or via batchEmbedding-style APIs) when tokenCountEstimator.estimate(formattedContent) for any one Document returns a value greater than maxInputTokenCount.

Common situations: Very large documents ingested without chunking (e.g. whole PDFs or long transcripts); configuring a low maxInputTokenCount (or a low model token limit) while feeding large docs; a token count estimator undercounting/overcounting relative to the actual embedding model.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

AI-assisted analysis of spring-projects/spring-ai@98a7beda4f (2026-09-11). Data as JSON: /api/errors/18eab4d460fd46bb. Report an issue: GitHub.