{"record":{"id":"d840d4ea3a812855","repo":"microsoft/semantic-kernel","slug":"batch-upsert-failed","errorCode":null,"errorMessage":"Batch upsert failed","messagePattern":"Batch upsert failed","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/semantic_kernel/connectors/memory_stores/mongodb_atlas/mongodb_atlas_memory_store.py","lineNumber":193,"sourceCode":"        Returns:\n            List[str]: The unique identifiers for the memory records.\n        \"\"\"\n        upserts: list[UpdateOne] = []\n        for record in records:\n            document = memory_record_to_mongo_document(record)\n            upserts.append(UpdateOne(document, {\"$set\": document}, upsert=True))\n        bulk_update_result: results.BulkWriteResult = await self.database[collection_name].bulk_write(\n            upserts, ordered=False\n        )\n\n        # Assert the number matched and the number upserted equal the total batch updated\n        logger.debug(\n            \"matched_count=%s, upserted_count=%s\",\n            bulk_update_result.matched_count,\n            bulk_update_result.upserted_count,\n        )\n        if bulk_update_result.matched_count + bulk_update_result.upserted_count != len(records):\n            raise ValueError(\"Batch upsert failed\")\n        return [record._id for record in records]\n\n    async def get(self, collection_name: str, key: str, with_embedding: bool) -> MemoryRecord:\n        \"\"\"Gets a memory record from the data store. Does not guarantee that the collection exists.\n\n        Args:\n            collection_name (str): The name associated with a collection of embeddings.\n            key (str): The unique id associated with the memory record to get.\n            with_embedding (bool): If true, the embedding will be returned in the memory record.\n\n        Returns:\n            MemoryRecord: The memory record if found\n        \"\"\"\n        document = await self.database[collection_name].find_one({MONGODB_FIELD_ID: key})\n\n        return document_to_memory_record(document, with_embedding) if document else None\n\n    async def get_batch(self, collection_name: str, keys: list[str], with_embeddings: bool) -> list[MemoryRecord]:","sourceCodeStart":175,"sourceCodeEnd":211,"githubUrl":"https://github.com/microsoft/semantic-kernel/blob/c028a0c7dc4f0814cdcbaba9d998f187a41197bf/python/semantic_kernel/connectors/memory_stores/mongodb_atlas/mongodb_atlas_memory_store.py#L175-L211","documentation":"Raised as a ValueError in MongoDBAtlasMemoryStore.upsert_batch when the sum of matched_count and upserted_count from the bulk_write result does not equal the number of records submitted. The store uses bulk_write with ordered=False and UpdateOne(upsert=True); each record should match or upsert exactly once, so a mismatch indicates some records were neither matched nor inserted.","triggerScenarios":"A bulk write where some operations fail validation server-side (e.g. document schema validation rules in Atlas), are dropped, or produce duplicate-key conflicts in an unordered bulk. The counts diverge from the input batch size.","commonSituations":"MongoDB collection-level JSON schema validation rejecting some documents. Duplicate _id conflicts causing some UpdateOne ops to not count as matched or upserted. Network issues causing partial bulk completion. Documents missing required fields per a validator.","solutions":["Inspect the BulkWriteResult (matched_count, upserted_count, write_errors) by performing the bulk_write manually to see which documents failed.","Check for collection-level schema validation rules that reject some records.","Ensure each record produces a unique, valid document with a well-formed _id.","Reduce the batch and retry to isolate the offending record(s)."],"exampleFix":"// before\nids = await store.upsert_batch('docs', records)  # ValueError: Batch upsert failed\n// after\ntry:\n    ids = await store.upsert_batch('docs', records)\nexcept ValueError:\n    # fall back to single upserts to isolate failures\n    ids = []\n    for r in records:\n        try:\n            ids.append(await store.upsert('docs', r))\n        except Exception as e:\n            logging.error('Record upsert failed: %s', e)","handlingStrategy":"try-catch","validationCode":"def documents_have_unique_ids(records) -> bool:\n    ids = [r._id for r in records]\n    return len(ids) == len(set(ids))\n\nif not documents_have_unique_ids(records):\n    raise ValueError('Duplicate ids in batch')","typeGuard":null,"tryCatchPattern":"try:\n    ids = await store.upsert_batch('docs', records)\nexcept ValueError:\n    # isolate failing records by falling back to single upserts\n    ids = []\n    for r in records:\n        try:\n            ids.append(await store.upsert('docs', r))\n        except Exception as e:\n            logging.error('Record failed: %s', e)","preventionTips":["Ensure each record has a unique, valid _id.","Check for collection-level schema validation rules that reject documents.","Inspect BulkWriteResult write_errors by testing bulk_write directly.","Reduce batch size to isolate offending records."],"tags":["mongodb-atlas","upsert","bulk-write","data-integrity","python"],"backgroundTag":null,"analyzedSha":"c028a0c7dc4f0814cdcbaba9d998f187a41197bf","analyzedAt":"2026-08-13T13:48:05.040Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}