{"record":{"id":"18eab4d460fd46bb","repo":"spring-projects/spring-ai","slug":"tokens-in-a-single-document-exceeds-the-maximum-nu","errorCode":null,"errorMessage":"Tokens in a single document exceeds the maximum number of allowed input tokens","messagePattern":"Tokens in a single document exceeds the maximum number of allowed input tokens","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spring-ai-model/src/main/java/org/springframework/ai/embedding/TokenCountBatchingStrategy.java","lineNumber":148,"sourceCode":"\t\tthis.tokenCountEstimator = tokenCountEstimator;\n\t\tthis.maxInputTokenCount = (int) Math.round(maxInputTokenCount * (1 - reservePercentage));\n\t\tthis.contentFormatter = contentFormatter;\n\t\tthis.metadataMode = metadataMode;\n\t}\n\n\t@Override\n\tpublic List<List<Document>> batch(List<Document> documents) {\n\t\tList<List<Document>> batches = new ArrayList<>();\n\t\tint currentSize = 0;\n\t\tList<Document> currentBatch = new ArrayList<>();\n\n\t\t// Do not collect the documents into a Map keyed by Document: equal documents\n\t\t// would collapse to a single entry and be silently dropped from the batches.\n\t\tfor (Document document : documents) {\n\t\t\tint tokenCount = this.tokenCountEstimator\n\t\t\t\t.estimate(document.getFormattedContent(this.contentFormatter, this.metadataMode));\n\t\t\tif (tokenCount > this.maxInputTokenCount) {\n\t\t\t\tthrow new IllegalArgumentException(\n\t\t\t\t\t\t\"Tokens in a single document exceeds the maximum number of allowed input tokens\");\n\t\t\t}\n\t\t\tcurrentSize += tokenCount;\n\t\t\tif (currentSize > this.maxInputTokenCount) {\n\t\t\t\tbatches.add(currentBatch);\n\t\t\t\tcurrentBatch = new ArrayList<>();\n\t\t\t\tcurrentSize = tokenCount;\n\t\t\t}\n\t\t\tcurrentBatch.add(document);\n\t\t}\n\t\tif (!currentBatch.isEmpty()) {\n\t\t\tbatches.add(currentBatch);\n\t\t}\n\t\treturn batches;\n\t}\n\n}\n","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/spring-projects/spring-ai/blob/98a7beda4f29d80a71c5837eb4053b03a93a46f7/spring-ai-model/src/main/java/org/springframework/ai/embedding/TokenCountBatchingStrategy.java#L130-L166","documentation":"TokenCountBatchingStrategy.batch() splits documents into batches whose total estimated token count stays under maxInputTokenCount. If a single document alone exceeds that maximum, it can never fit in a valid batch, so the strategy throws IllegalArgumentException instead of producing a batch that the model would reject.","triggerScenarios":"Calling batch(docs) (directly or via batchEmbedding-style APIs) when tokenCountEstimator.estimate(formattedContent) for any one Document returns a value greater than maxInputTokenCount.","commonSituations":"Very large documents ingested without chunking (e.g. whole PDFs or long transcripts); configuring a low maxInputTokenCount (or a low model token limit) while feeding large docs; a token count estimator undercounting/overcounting relative to the actual embedding model.","solutions":["Split large documents into smaller chunks (e.g. TokenTextSplitter) before passing them to the embedding model","Raise TokenCountBatchingStrategy.maxInputTokenCount via its constructor/builder to match your embedding model's actual limit","Shorten document content or reduce metadata included in formattedContent (adjust ContentFormatter/MetadataMode) to lower the estimated token count"],"exampleFix":"// before\nembeddingModel.embed(documents); // one doc exceeds max tokens\n// after\nList<Document> chunks = new TokenTextSplitter().apply(documents);\nembeddingModel.embed(chunks);","handlingStrategy":"validation","validationCode":"TokenCountBatchingStrategy strategy = new TokenCountBatchingStrategy();\nfor (Document doc : documents) {\n    int tokens = strategy.getTokenCountEstimator()\n        .estimate(doc.getFormattedContent());\n    if (tokens > strategy.getMaxInputTokenCount()) {\n        throw new IllegalStateException(\"Document too large: \" + doc.getId() + \" tokens=\" + tokens);\n    }\n}","typeGuard":null,"tryCatchPattern":"try {\n    embeddingModel.embed(documents);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().contains(\"exceeds the maximum number of allowed input tokens\")) {\n        List<Document> chunks = new TokenTextSplitter().apply(documents);\n        embeddingModel.embed(chunks);\n    } else {\n        throw e;\n    }\n}","preventionTips":["Always chunk large documents with TokenTextSplitter before embedding","Set maxInputTokenCount to your embedding model's documented input limit with headroom","Keep an eye on formattedContent size — metadata included via ContentFormatter counts toward tokens"],"tags":["embedding","token-limit","batching","document-too-large"],"backgroundTag":"payload-too-large","analyzedSha":"98a7beda4f29d80a71c5837eb4053b03a93a46f7","analyzedAt":"2026-09-11T14:15:49.441Z","contentChangedAt":"2026-09-11T14:15:49.441Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}