{"record":{"id":"324f4fc945b43062","repo":"spring-projects/spring-ai","slug":"failed-to-delete-some-documents","errorCode":null,"errorMessage":"Failed to delete some documents: ","messagePattern":"Failed to delete some documents: ","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"vector-stores/spring-ai-opensearch-store/src/main/java/org/springframework/ai/vectorstore/opensearch/OpenSearchVectorStore.java","lineNumber":293,"sourceCode":"\t@Override\n\tprotected void doDelete(Filter.Expression filterExpression) {\n\t\tAssert.notNull(filterExpression, \"Filter expression must not be null\");\n\n\t\ttry {\n\t\t\tString filterStr = this.filterExpressionConverter.convertExpression(filterExpression);\n\n\t\t\t// Create delete by query request\n\t\t\tDeleteByQueryRequest request = new DeleteByQueryRequest.Builder().index(this.index)\n\t\t\t\t.query(q -> q.queryString(qs -> qs.query(filterStr)))\n\t\t\t\t.build();\n\n\t\t\tDeleteByQueryResponse response = this.openSearchClient.deleteByQuery(request);\n\t\t\tif (logger.isDebugEnabled()) {\n\t\t\t\tlogger.debug(\"Deleted \" + response.deleted() + \" documents matching filter expression\");\n\t\t\t}\n\n\t\t\tif (!response.failures().isEmpty()) {\n\t\t\t\tthrow new IllegalStateException(\"Failed to delete some documents: \" + response.failures());\n\t\t\t}\n\t\t}\n\t\tcatch (Exception e) {\n\t\t\tif (logger.isErrorEnabled()) {\n\t\t\t\tlogger.error(\"Failed to delete documents by filter: \" + e.getMessage());\n\t\t\t}\n\t\t\tthrow new IllegalStateException(\"Failed to delete documents by filter\", e);\n\t\t}\n\t}\n\n\t@Override\n\tpublic List<Document> doSimilaritySearch(SearchRequest searchRequest) {\n\t\tAssert.notNull(searchRequest, \"The search request must not be null.\");\n\t\treturn similaritySearch(this.embeddingModel.embed(searchRequest.getQuery()), searchRequest.getTopK(),\n\t\t\t\tsearchRequest.getSimilarityThreshold(), searchRequest.getFilterExpression());\n\t}\n\n\tpublic List<Document> similaritySearch(float[] embedding, int topK, double similarityThreshold,","sourceCodeStart":275,"sourceCodeEnd":311,"githubUrl":"https://github.com/spring-projects/spring-ai/blob/98a7beda4f29d80a71c5837eb4053b03a93a46f7/vector-stores/spring-ai-opensearch-store/src/main/java/org/springframework/ai/vectorstore/opensearch/OpenSearchVectorStore.java#L275-L311","documentation":"OpenSearchVectorStore.doDelete, when deleting by filter, executes a delete_by_query; if the DeleteByQueryResponse contains failures, it throws IllegalStateException('Failed to delete some documents: ' + failures). This indicates a partial delete — some documents matching the filter could not be removed.","triggerScenarios":"Calling vectorStore.delete(Filter.Expression) or delete(String filterExpression) where the delete_by_query partially fails, e.g. due to document-level conflicts, version conflicts from concurrent updates, or mapping/field issues.","commonSituations":"Concurrent indexing while deleting by filter (version conflicts); cluster under load with timeouts on some shards; indices with conflicting mappings for metadata fields used in the filter.","solutions":["Read the failures list in the message to identify failing shards/documents and reasons.","Retry the delete_by_query with conflicts=proceed semantics or during a maintenance window without concurrent writes.","Check cluster health and shard allocation; resolve red/yellow shards before re-running.","Validate the filter field mappings (metadata_ prefix) are consistent across documents."],"exampleFix":"// before\nvectorStore.delete(filterExpression); // partial failures during heavy indexing\n// after\n// pause ingestion, then retry\ningestionPaused = true;\nvectorStore.delete(filterExpression); // conflicts eliminated\ningestionPaused = false;","handlingStrategy":"retry","validationCode":"Health health = client.cluster().health(h -> h); // proceed only if status != RED\nif (health.status() == Status.RED) { /* defer delete */ }","typeGuard":null,"tryCatchPattern":"try {\n    vectorStore.delete(filterExpression);\n} catch (IllegalStateException e) {\n    // parse failures from message; retry with conflicts-proceed during quiet period\n    scheduler.schedule(() -> vectorStore.delete(filterExpression), 30, TimeUnit.SECONDS);\n}","preventionTips":["Avoid delete-by-filter while concurrent indexing is running","Check cluster/shard health before large deletes","Parse the failures list to target retries at failing shards only","Ensure metadata field mappings are consistent across documents"],"tags":["opensearch","delete-by-query","partial-failure"],"backgroundTag":"database-query-failed","analyzedSha":"98a7beda4f29d80a71c5837eb4053b03a93a46f7","analyzedAt":"2026-09-11T14:15:49.441Z","contentChangedAt":"2026-09-11T14:15:49.441Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}