{"record":{"id":"73aea1e28ab986d0","repo":"agentscope-ai/agentscope","slug":"elasticsearch-bulk-insert-failed-for-len-failures","errorCode":null,"errorMessage":"Elasticsearch bulk insert failed for {len(failures)} record(s)","messagePattern":"Elasticsearch bulk insert failed for (.+?) record\\(s\\)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/agentscope/rag/_vdb/_elasticsearch.py","lineNumber":157,"sourceCode":"                        \"vector\": record.vector,\n                        \"document_id\": record.document_id,\n                        \"chunk\": record.chunk.model_dump(mode=\"json\"),\n                        \"metadata\": record.chunk.metadata,\n                    },\n                ],\n            )\n\n        response = await self.get_client().bulk(\n            operations=operations,\n            refresh=self._refresh,\n        )\n        if response.get(\"errors\"):\n            failures = [\n                item\n                for item in response.get(\"items\", [])\n                if next(iter(item.values())).get(\"error\")\n            ]\n            raise RuntimeError(\n                f\"Elasticsearch bulk insert failed for {len(failures)} \"\n                \"record(s)\",\n            )\n\n    async def delete(self, collection: str, document_id: str) -> None:\n        \"\"\"Delete every chunk belonging to one source document.\"\"\"\n        await self.get_client().delete_by_query(\n            index=collection,\n            query={\"term\": {\"document_id\": document_id}},\n            conflicts=\"proceed\",\n            refresh=self._refresh is not False,\n        )\n\n    async def search(\n        self,\n        collection: str,\n        query_vector: list[float],\n        top_k: int = 5,","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/agentscope-ai/agentscope/blob/e90f1c7592896cc95f6e5ee506194f533378247d/src/agentscope/rag/_vdb/_elasticsearch.py#L139-L175","documentation":"After a bulk _bulk API call, Elasticsearch reported partial errors; the store counts failing items and raises RuntimeError, aborting the insert even if some records succeeded.","triggerScenarios":"insert() with malformed records (bad vector dimension vs index mapping), oversized payloads, or cluster issues (disk watermark exceeded, mapper parsing exceptions) causing per-item errors in the bulk response.","commonSituations":"Embedding model changed so vectors no longer match the index dimension; indexing after switching embedding providers without recreating the collection.","solutions":["Check the ES logs / rerun with response inspection to see item['index']['error'] reasons","Verify vector dimension matches the index mapping; recreate collection if the embedding model changed","Reduce batch size if payloads exceed limits","Free disk space / resolve cluster watermark issues"],"exampleFix":"# before\nvdb = ElasticsearchVectorDatabase(..., dim=384)\nvdb.insert(collection, records_768_dim)\n# after\nvdb = ElasticsearchVectorDatabase(..., dim=768)\nvdb.insert(collection, records_768_dim)","handlingStrategy":"try-catch","validationCode":"assert all(len(r['vector']) == expected_dim for r in records)","typeGuard":"def dims_match(records, dim: int) -> bool:\n    return all(len(r['vector']) == dim for r in records)","tryCatchPattern":"try:\n    await vdb.insert(coll, records)\nexcept RuntimeError as e:\n    if 'bulk insert failed' in str(e):\n        logger.error('partial bulk failure; check item errors and index mapping')\n    raise","preventionTips":["Pin one embedding model per collection; recreate collections after model changes","Validate vector dimensions against the index mapping before bulk insert","Keep ES disk usage below watermarks"],"tags":["elasticsearch","bulk-insert","vector-dimension","rag"],"backgroundTag":"bulk-index-partial-failure","analyzedSha":"e90f1c7592896cc95f6e5ee506194f533378247d","analyzedAt":"2026-08-28T18:24:12.087Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}