{"record":{"id":"053c58dd34c02833","repo":"666ghj/MiroFish","slug":"zep-batch-submission-batch-id-contains-len-item","errorCode":null,"errorMessage":"Zep batch {submission.batch_id} contains {len(items)} items, expected {submission.item_count}","messagePattern":"Zep batch (.+?) contains (.+?) items, expected (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"backend/app/services/graph_builder.py","lineNumber":685,"sourceCode":"                )\n\n            if status in terminal_states:\n                break\n            time.sleep(3)\n\n        items = self._list_batch_items(submission.batch_id)\n        if status != \"succeeded\":\n            failed_items = [\n                item for item in items\n                if getattr(item, \"status\", None) not in {\"succeeded\", \"skipped\"}\n            ]\n            first_error = getattr(failed_items[0], \"error\", None) if failed_items else None\n            raise RuntimeError(\n                f\"Zep batch {submission.batch_id} ended as {status}; \"\n                f\"failed_items={len(failed_items)}; first_error={first_error}\"\n            )\n        if len(items) != submission.item_count:\n            raise RuntimeError(\n                f\"Zep batch {submission.batch_id} contains {len(items)} items, \"\n                f\"expected {submission.item_count}\"\n            )\n\n        ordered_items = sorted(\n            items,\n            key=lambda item: getattr(item, \"sequence_index\", 0) or 0,\n        )\n        episode_uuids: List[str] = []\n        for item in ordered_items:\n            item_status = getattr(item, \"status\", None)\n            episode_uuid = getattr(item, \"episode_uuid\", None)\n            source_uuid = getattr(item, \"source_uuid\", None)\n            if item_status != \"succeeded\" or not episode_uuid:\n                raise RuntimeError(\n                    f\"Zep batch {submission.batch_id} returned an incomplete item\"\n                )\n            if source_uuid and source_uuid != episode_uuid:","sourceCodeStart":667,"sourceCodeEnd":703,"githubUrl":"https://github.com/666ghj/MiroFish/blob/b5b53acc57189a4a42e44a23e149dc655c98fe82/backend/app/services/graph_builder.py#L667-L703","documentation":"Raised by GraphBuilder._wait_for_batch when the batch status is 'succeeded' but paginating all items via _list_batch_items returns a count different from submission.item_count. It guards against silent item loss: a succeeded batch must yield exactly the number of items that were submitted, otherwise downstream UUID ordering would be wrong. A mismatch means pagination dropped/duplicated pages or the submission bookkeeping miscounted.","triggerScenarios":"Cursor pagination in _list_batch_items skipping or repeating pages for batches over 100 items; items still propagating when listed immediately after 'succeeded' (eventual consistency); submitting items in multiple add calls but recording the wrong item_count in BatchSubmission.","commonSituations":"Large batches right at the moment of completion; Zep cursor anomalies (also visible as error 21); code paths that add items in a loop but construct BatchSubmission with a stale count; retries of the add call that double-append items without updating item_count.","solutions":["Re-list the batch items after a short delay — propagation lag right after 'succeeded' often resolves the count.","Verify how item_count is computed where BatchSubmission is built; it must equal the exact number of items sent in the add call(s).","Compare counts per page of list_items to spot skipped/duplicated cursors; if cursors misbehave, see error 21's guidance (SDK version, retry).","If the count is consistently wrong on Zep's side, escalate with batch_id — silent item loss is a service defect."],"exampleFix":"# before\nitems = self._list_batch_items(submission.batch_id)\n# after (allow a short propagation window)\nitems = self._list_batch_items(submission.batch_id)\nfor _ in range(3):\n    if len(items) == submission.item_count:\n        break\n    time.sleep(5)\n    items = self._list_batch_items(submission.batch_id)","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    builder._wait_for_batch(submission)\nexcept RuntimeError as e:\n    if 'items, expected' in str(e):\n        time.sleep(10)\n        items = builder._list_batch_items(submission.batch_id)  # re-read once for propagation lag\n        if len(items) != submission.item_count:\n            raise\n    else:\n        raise","preventionTips":["Derive BatchSubmission.item_count from the same list passed to the add call — never a separate counter.","When retrying an add after ambiguity, reconcile the actual item count server-side before proceeding.","Keep pagination cursor logs to audit for skipped pages."],"tags":["zep","batch-api","pagination","data-integrity","ingestion"],"backgroundTag":null,"analyzedSha":"b5b53acc57189a4a42e44a23e149dc655c98fe82","analyzedAt":"2026-08-14T22:29:33.146Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}