{"record":{"id":"2a5046f7d03c6d61","repo":"unslothai/unsloth","slug":"linked-source-changed-while-it-was-copied","errorCode":null,"errorMessage":"Linked source changed while it was copied","messagePattern":"Linked source changed while it was copied","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"warning","filePath":"studio/backend/core/rag/folder_sync.py","lineNumber":1052,"sourceCode":"            metadata[\"inode\"],\n        )\n        actual = (before.st_size, before.st_mtime_ns, before.st_dev, before.st_ino)\n        # Only an identity os.fstat can reproduce is comparable here: os.scandir reports none at all\n        # on Windows, and the os.lstat _scan falls back to disagrees with os.fstat on file systems\n        # without stable file ids. Size and mtime still gate those, and the post-copy check below is\n        # fstat to fstat either way.\n        usable = metadata[\"inode\"] not in (None, 0) and not metadata.get(\"identity_from_path\")\n        compared = 4 if usable else 2\n        if actual[:compared] != expected[:compared]:\n            raise RuntimeError(\"Linked source changed during reconciliation\")\n        if config.MAX_UPLOAD_BYTES and before.st_size > config.MAX_UPLOAD_BYTES:\n            raise RuntimeError(\"Linked source exceeds the RAG file size limit\")\n        with os.fdopen(fd, \"rb\", closefd = False) as src, open(target, \"xb\") as dst:\n            _copy_exact(src, dst, before.st_size)\n        after = os.fstat(fd)\n        # Both sides are fstat here, so the identity is always comparable.\n        if (after.st_size, after.st_mtime_ns, after.st_dev, after.st_ino) != actual:\n            raise RuntimeError(\"Linked source changed while it was copied\")\n        return str(target)\n    except Exception:\n        if target is not None:\n            _remove_snapshot(str(target))\n        raise\n    finally:\n        os.close(fd)\n\n\ndef _copy_exact(source, target, size: int) -> None:\n    remaining = size\n    while remaining:\n        block = source.read(min(1 << 20, remaining))\n        if not block:\n            raise RuntimeError(\"Linked source changed while it was copied\")\n        target.write(block)\n        remaining -= len(block)\n    if source.read(1):","sourceCodeStart":1034,"sourceCodeEnd":1070,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/rag/folder_sync.py#L1034-L1070","documentation":"After _copy_exact finishes, _snapshot fstats the still-open source descriptor again and compares (size, mtime_ns, dev, ino) to the pre-copy values. Any drift means the file was modified while it was being read, so the completed snapshot could be torn and is deleted via _remove_snapshot before the error propagates.","triggerScenarios":"The source file is written, truncated, or atomically replaced (new inode) during the copy loop itself; large files make the copy window long enough for concurrent writers (logs, exporters, editors) to interleave.","commonSituations":"Syncing directories that another process actively writes (database export dirs, download folders, live log trees); slow network shares stretching copy time into write windows.","solutions":["Quiesce writers on the linked folder and re-run sync — the next pass snapshots the settled file.","Sync on a schedule when applications are idle; exclude actively-written directories from the link.","For log-like data, snapshot from rotated (closed) files rather than the live one."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"import os\n\ndef file_still_settled(path: str, size: int, mtime_ns: int) -> bool:\n    st = os.stat(path)\n    return st.st_size == size and st.st_mtime_ns == mtime_ns","typeGuard":null,"tryCatchPattern":"try:\n    reconcile(folder_id)\nexcept RuntimeError as e:\n    if \"changed while it was copied\" in str(e):\n        wait_and_resync(folder_id)  # snapshot of the settled file lands next cycle\n    else:\n        raise","preventionTips":["Quiesce writers before triggering sync on linked folders.","Sync rotated/closed files instead of live ones for log-like data.","On slow network shares, copy hot files locally first and link that copy."],"tags":["rag","folder-sync","toctou","concurrent-writes","copy"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}