{"record":{"id":"0dffd5b2d22d74b7","repo":"bytedance/deer-flow","slug":"duplicate-fact","errorCode":null,"errorMessage":"Duplicate fact","messagePattern":"Duplicate fact","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py","lineNumber":470,"sourceCode":"        return None\n    stripped = content.strip()\n    if not stripped:\n        return None\n    return stripped.casefold()\n\n\ndef _raise_if_duplicate_fact_content(memory_data: dict[str, Any], content_key: str | None) -> None:\n    \"\"\"Reject a candidate fact whose normalized content already exists.\n\n    Callers must invoke this against the freshest snapshot available inside\n    their read-check-write critical section (i.e. on every revision-conflict\n    retry), so two concurrent creators of the same content cannot both pass\n    the check and store duplicate facts.\"\"\"\n    if content_key is None:\n        return\n    for fact in memory_data.get(\"facts\", []):\n        if isinstance(fact, dict) and _fact_content_key(fact.get(\"content\")) == content_key:\n            raise ValueError(\"Duplicate fact\")\n\n\n# ── Staleness review helpers ──────────────────────────────────────────────\n\n\ndef _parse_fact_datetime(raw: str) -> datetime | None:\n    \"\"\"Parse an ISO-8601 datetime string from a fact's createdAt field.\n\n    Returns ``None`` on any parse failure so callers can safely skip malformed facts.\n    \"\"\"\n    if not raw:\n        return None\n    try:\n        result = datetime.fromisoformat(raw)\n        # Naive datetimes (no tzinfo) would cause TypeError when compared\n        # with the timezone-aware cutoff.  Assume UTC for safety.\n        if result.tzinfo is None:\n            result = result.replace(tzinfo=UTC)","sourceCodeStart":452,"sourceCodeEnd":488,"githubUrl":"https://github.com/bytedance/deer-flow/blob/1dd6ba1acb03700589994b0366c5d1c7d05e2eff/backend/packages/harness/deerflow/agents/memory/backends/deermem/deermem/core/updater.py#L452-L488","documentation":"_raise_if_duplicate_fact_content() rejects a candidate fact whose normalized content already exists in the current snapshot; _fact_content_key() normalizes the content string so trivial differences (whitespace/case, per its normalization) still collide. It is designed to be re-run inside the read-check-write critical section on every revision-conflict retry, so two concurrent creators of the same content cannot both store it.","triggerScenarios":"create_memory_fact called with content the agent already stored (e.g. the same preference re-extracted on a later turn); concurrent extraction of the same fact by two threads; content differing only in ways _fact_content_key normalizes away.","commonSituations":"Memory extraction running on every turn without dedup awareness; re-importing an exported memory file; retrying a user request that re-creates the same fact; users restating the same information phrased identically.","solutions":["Treat the error as 'already known': catch ValueError, check that str(e) == 'Duplicate fact', and return the existing fact (lookup via list_facts with the content key) instead of failing the request.","Before creating, query for the normalized content and update the existing fact's revision/metadata instead of inserting.","Debounce or dedupe extraction runs so identical extractions within a window collapse into one call."],"exampleFix":"# before\nupdater.create_memory_fact(agent_name=a, content=\"User lives in Berlin\", confidence=0.9)\n\n# after\ntry:\n    updater.create_memory_fact(agent_name=a, content=\"User lives in Berlin\", confidence=0.9)\nexcept ValueError as exc:\n    if str(exc) != \"Duplicate fact\":\n        raise\n    # already stored; refresh it instead\n    facts = storage.list_facts(agent_name=a, filters={})\n    existing = next(f for f in facts if normalize(f[\"content\"]) == normalize(\"User lives in Berlin\"))\n    storage.upsert_fact({**existing, \"confidence\": 0.9}, agent_name=a)","handlingStrategy":"try-catch","validationCode":"existing = [f for f in storage.list_facts(agent_name=agent_name) if _fact_content_key(f.get(\"content\")) == _fact_content_key(new_content)]\nif existing:\n    return existing[0]  # skip create, update instead","typeGuard":null,"tryCatchPattern":"try:\n    updater.create_memory_fact(agent_name=a, content=content, confidence=c)\nexcept ValueError as exc:\n    if str(exc) != \"Duplicate fact\":\n        raise\n    logger.info(\"fact already present; updating instead\")\n    # fall through to an update path on the existing fact","preventionTips":["Check for existing normalized content before creating a fact.","Treat 'Duplicate fact' as an idempotent success, not a failure, in extraction pipelines.","Debounce repeated memory extraction of the same turn."],"tags":["memory","deduplication","business-rule","deermem"],"backgroundTag":null,"analyzedSha":"1dd6ba1acb03700589994b0366c5d1c7d05e2eff","analyzedAt":"2026-08-14T21:20:34.804Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}