{"record":{"id":"6739ed6b66844318","repo":"apache/beam","slug":"err-re-raised-oserror-from-os-rename","errorCode":null,"errorMessage":"err (re-raised OSError from os.rename)","messagePattern":"err \\(re-raised OSError from os\\.rename\\)","errorType":"exception","errorClass":"IOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/localfilesystem.py","lineNumber":243,"sourceCode":"\n    Args:\n      source_file_names: List of file paths that need to be moved\n      destination_file_names: List of destination_file_names for the files\n\n    Raises:\n      ``BeamIOError``: if any of the rename operations fail\n    \"\"\"\n    err_msg = (\n        \"source_file_names and destination_file_names should \"\n        \"be equal in length\")\n    assert len(source_file_names) == len(destination_file_names), err_msg\n\n    def _rename_file(source, destination):\n      \"\"\"Rename a single file object\"\"\"\n      try:\n        os.rename(source, destination)\n      except OSError as err:\n        raise IOError(err)\n\n    exceptions = {}\n    for source, destination in zip(source_file_names, destination_file_names):\n      try:\n        _rename_file(source, destination)\n      except Exception as e:  # pylint: disable=broad-except\n        exceptions[(source, destination)] = e\n\n    if exceptions:\n      raise BeamIOError(\"Rename operation failed\", exceptions)\n\n  def exists(self, path):\n    \"\"\"Check if the provided path exists on the FileSystem.\n\n    Args:\n      path: string path that needs to be checked.\n\n    Returns: boolean flag indicating if path exists","sourceCodeStart":225,"sourceCodeEnd":261,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/localfilesystem.py#L225-L261","documentation":"Inside LocalFileSystem.rename, the helper _rename_file performs os.rename for a single path pair and converts any OSError into IOError(err). Typical causes are a missing source file (FileNotFoundError) or cross-filesystem rename attempts (Invalid cross-device link, EXDEV), since os.rename cannot move across mount points. The per-file IOError is then aggregated by rename into a BeamIOError.","triggerScenarios":"Renaming a file that does not exist; renaming between different filesystems/mount points (e.g. /tmp to /home on separate devices) which raises 'Invalid cross-device link'; renaming onto a destination in a directory without write permission; renaming a directory into a subdirectory of itself.","commonSituations":"Moving staging output from a tmpfs temp dir to a persistent volume; two workers renaming the same temp file concurrently; mis-typed paths where the source was never created.","solutions":["Check the underlying error in the chained IOError: ENOENT means fix the source path, EXDEV means copy+delete instead of rename.","Use shutil.move semantics (copy then remove) when source and destination are on different filesystems.","Verify the source exists (FileSystems.exists / os.path.exists) before renaming and that the destination parent exists and is writable.","Guard against concurrent workers renaming the same file with unique per-worker temp names.","Catch the eventual BeamIOError from rename and inspect exception_details for the failing pair."],"exampleFix":"// before\nFileSystems.rename(['/tmp/stage/out'], ['/data/out'])  # EXDEV if mounts differ\n// after\nimport shutil\nif os.path.exists('/tmp/stage/out'):\n    shutil.move('/tmp/stage/out', '/data/out')  # falls back to copy across devices","handlingStrategy":"validation","validationCode":"import os\nif os.path.exists(src) and os.path.exists(os.path.dirname(dst)):\n    os.rename(src, dst)","typeGuard":null,"tryCatchPattern":"try:\n    FileSystems.rename([src], [dst])\nexcept Exception as e:\n    # EXDEV / missing source -> fall back to copy+delete\n    import shutil\n    shutil.move(src, dst)","preventionTips":["Avoid renaming across filesystems/mounts; use copy+delete (shutil.move) there","Check source existence before rename and use unique temp names to avoid worker races","Ensure the destination parent directory exists and is writable"],"tags":["python","apache-beam","filesystem","rename","oserror"],"backgroundTag":"file-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}