{"record":{"id":"51583337d23ef7f1","repo":"unslothai/unsloth","slug":"linked-source-changed-during-reconciliation","errorCode":null,"errorMessage":"Linked source changed during reconciliation","messagePattern":"Linked source changed during reconciliation","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"warning","filePath":"studio/backend/core/rag/folder_sync.py","lineNumber":1044,"sourceCode":"        target = ensure_dir(rag_uploads_root()) / f\"linked-{uuid.uuid4().hex}{ext}\"\n        before = os.fstat(fd)\n        if not stat.S_ISREG(before.st_mode):\n            raise RuntimeError(\"Linked source is not a regular file\")\n        expected = (\n            metadata[\"size_bytes\"],\n            metadata[\"mtime_ns\"],\n            metadata[\"device\"],\n            metadata[\"inode\"],\n        )\n        actual = (before.st_size, before.st_mtime_ns, before.st_dev, before.st_ino)\n        # Only an identity os.fstat can reproduce is comparable here: os.scandir reports none at all\n        # on Windows, and the os.lstat _scan falls back to disagrees with os.fstat on file systems\n        # without stable file ids. Size and mtime still gate those, and the post-copy check below is\n        # fstat to fstat either way.\n        usable = metadata[\"inode\"] not in (None, 0) and not metadata.get(\"identity_from_path\")\n        compared = 4 if usable else 2\n        if actual[:compared] != expected[:compared]:\n            raise RuntimeError(\"Linked source changed during reconciliation\")\n        if config.MAX_UPLOAD_BYTES and before.st_size > config.MAX_UPLOAD_BYTES:\n            raise RuntimeError(\"Linked source exceeds the RAG file size limit\")\n        with os.fdopen(fd, \"rb\", closefd = False) as src, open(target, \"xb\") as dst:\n            _copy_exact(src, dst, before.st_size)\n        after = os.fstat(fd)\n        # Both sides are fstat here, so the identity is always comparable.\n        if (after.st_size, after.st_mtime_ns, after.st_dev, after.st_ino) != actual:\n            raise RuntimeError(\"Linked source changed while it was copied\")\n        return str(target)\n    except Exception:\n        if target is not None:\n            _remove_snapshot(str(target))\n        raise\n    finally:\n        os.close(fd)\n\n\ndef _copy_exact(source, target, size: int) -> None:","sourceCodeStart":1026,"sourceCodeEnd":1062,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/rag/folder_sync.py#L1026-L1062","documentation":"Before copying, _snapshot compares the fstat of the opened descriptor against the metadata recorded during _scan (size, mtime, plus device/inode when comparable — see the identity_from_path caveat in the source). A mismatch means the file's content or identity changed between scan and copy, so the snapshot would be an inconsistent mixture and is rejected.","triggerScenarios":"A file is appended to, truncated, touched, or rewritten (new inode) after _scan but before _snapshot processes it — typically during active write workloads on the linked folder (logs being written, editors saving, CI output).","commonSituations":"Linking a folder that applications actively write to; syncing while a big copy/download into the folder is in progress; editors using atomic replace (new inode) on every save.","solutions":["Wait for writes to finish and re-run sync; the next cycle reconciles the settled state.","Sync during quiescent windows (cron at night, post-build) rather than while files stream in.","Exclude hot directories (logs, downloads-in-progress) from the linked root."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"import os\n\ndef file_is_quiet(metadata: dict) -> bool:\n    st = os.stat(metadata[\"path\"])\n    return (st.st_size, st.st_mtime_ns) == (metadata[\"size_bytes\"], metadata[\"mtime_ns\"])","typeGuard":null,"tryCatchPattern":"try:\n    reconcile(folder_id)\nexcept RuntimeError as e:\n    if \"changed during reconciliation\" in str(e):\n        wait_for_writers_to_settle(folder_id)\n        reconcile(folder_id)  # next pass snapshots the settled file\n    else:\n        raise","preventionTips":["Sync linked folders only when writers (apps, downloads, exporters) are idle.","Exclude actively-written directories (logs, tmp, downloads) from the linked root.","Batch the heavy writes, then trigger sync explicitly."],"tags":["rag","folder-sync","toctou","scan","concurrent-writes"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}