{"record":{"id":"c1c4f6872c0df07f","repo":"apache/beam","slug":"rename-operation-failed-localfilesystem","errorCode":null,"errorMessage":"Rename operation failed","messagePattern":"Rename operation failed","errorType":"exception","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/localfilesystem.py","lineNumber":253,"sourceCode":"        \"be equal in length\")\n    assert len(source_file_names) == len(destination_file_names), err_msg\n\n    def _rename_file(source, destination):\n      \"\"\"Rename a single file object\"\"\"\n      try:\n        os.rename(source, destination)\n      except OSError as err:\n        raise IOError(err)\n\n    exceptions = {}\n    for source, destination in zip(source_file_names, destination_file_names):\n      try:\n        _rename_file(source, destination)\n      except Exception as e:  # pylint: disable=broad-except\n        exceptions[(source, destination)] = e\n\n    if exceptions:\n      raise BeamIOError(\"Rename operation failed\", exceptions)\n\n  def exists(self, path):\n    \"\"\"Check if the provided path exists on the FileSystem.\n\n    Args:\n      path: string path that needs to be checked.\n\n    Returns: boolean flag indicating if path exists\n    \"\"\"\n    return os.path.exists(path)\n\n  def size(self, path):\n    \"\"\"Get size of path on the FileSystem.\n\n    Args:\n      path: string path in question.\n\n    Returns: int size of path according to the FileSystem.","sourceCodeStart":235,"sourceCodeEnd":271,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/localfilesystem.py#L235-L271","documentation":"LocalFileSystem.rename renames each (source, destination) pair via _rename_file, collecting per-pair exceptions into a dict. If any pair failed, it raises BeamIOError('Rename operation failed', exceptions) whose exception_details maps each (source, destination) tuple to the underlying error. This is the batch-level failure signal for local renames in Beam's FileSystem API.","triggerScenarios":"Calling LocalFileSystem.rename or FileSystems.rename where at least one os.rename fails: source missing, cross-device move, or permission denied on the destination directory. One failing pair raises the aggregate error even though other pairs may have been renamed already.","commonSituations":"Batch commit of output files where one source was already renamed/removed by a prior run or concurrent worker; moving files from tmpfs to disk (EXDEV); destination directory not writable.","solutions":["Inspect BeamIOError.exception_details to find which pairs failed; note that successful renames are not rolled back.","Fix the specific cause: correct missing source paths, or replace rename with copy+delete for cross-device moves.","Make renames idempotent: skip pairs whose destination already exists from a previous partial run.","Check write permission on every destination directory before renaming.","Catch BeamIOError and retry only the failed pairs from exception_details."],"exampleFix":"// before\nFileSystems.rename(srcs, dsts)  # partial failures lost in one exception\n// after\nfrom apache_beam.io.filesystem import BeamIOError\ntry:\n    FileSystems.rename(srcs, dsts)\nexcept BeamIOError as e:\n    failed = list(e.exception_details.keys())\n    # retry only failed pairs after fixing their cause","handlingStrategy":"try-catch","validationCode":"import os\nassert all(os.path.exists(s) for s in srcs), \"some rename sources are missing\"","typeGuard":null,"tryCatchPattern":"from apache_beam.io.filesystem import BeamIOError\ntry:\n    FileSystems.rename(srcs, dsts)\nexcept BeamIOError as e:\n    failed = list(e.exception_details.keys())  # successful ones are NOT rolled back","preventionTips":["Make renames idempotent: skip pairs already renamed in prior partial runs","Verify sources exist and destination parents are writable before batching","Retry only failed pairs using exception_details"],"tags":["python","apache-beam","filesystem","rename","batch"],"backgroundTag":"file-write-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}