{"record":{"id":"c5c68845d8afed15","repo":"apache/beam","slug":"rename-operation-failed-gcsfilesystem","errorCode":null,"errorMessage":"Rename operation failed","messagePattern":"Rename operation failed","errorType":"exception","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/gcsfilesystem.py","lineNumber":272,"sourceCode":"    exceptions = {}\n    for batch in gcs_batches:\n      copy_statuses = self._gcsIO().copy_batch(batch)\n      copy_succeeded = {}\n      delete_targets = []\n      for src, dest, exception in copy_statuses:\n        if exception:\n          exceptions[(src, dest)] = exception\n        else:\n          copy_succeeded[src] = dest\n          delete_targets.append(src)\n      delete_statuses = self._gcsIO().delete_batch(delete_targets)\n      for src, exception in delete_statuses:\n        if exception:\n          dest = copy_succeeded[src]\n          exceptions[(src, dest)] = exception\n\n    if exceptions:\n      raise BeamIOError(\"Rename operation failed\", exceptions)\n\n  def exists(self, path):\n    \"\"\"Check if the provided path exists on the FileSystem.\n\n    Args:\n      path: string path that needs to be checked.\n\n    Returns: boolean flag indicating if path exists\n    \"\"\"\n    return self._gcsIO().exists(path)\n\n  def size(self, path):\n    \"\"\"Get size of path on the FileSystem.\n\n    Args:\n      path: string path in question.\n\n    Returns: int size of path according to the FileSystem.","sourceCodeStart":254,"sourceCodeEnd":290,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/gcsfilesystem.py#L254-L290","documentation":"GcsFileSystem.rename raises BeamIOError('Rename operation failed') when any copy or subsequent delete step of the rename fails. Rename is implemented as copy-then-delete; exceptions from either phase are aggregated per (source, destination) into one BeamIOError.","triggerScenarios":"Calling GcsFileSystem.rename(source_file_names, destination_file_names) where a copy fails (missing source, permissions) or the follow-up delete of the source fails, and then any exception is registered in the exceptions dict.","commonSituations":"Renaming files concurrently modified by another pipeline stage; source deleted between copy and delete; destination bucket quota or IAM issues.","solutions":["Read exception_details to see whether the copy or the delete phase failed and for which pair.","Ensure source files exist and are readable, and destination is writable before renaming.","Avoid concurrent mutations of the same objects during rename.","Re-run rename for the failed pairs after fixing the underlying GCS error."],"exampleFix":"// before\nfs.rename(srcs, dsts)\n// after\ntry:\n  fs.rename(srcs, dsts)\nexcept BeamIOError as e:\n  failed = [src for src, _ in e.exception_details]\n  # re-copy/rename only the failed sources","handlingStrategy":"try-catch","validationCode":"missing = [s for s in sources if not fs.exists(s)]\nif missing:\n    raise FileNotFoundError(f'missing sources: {missing}')","typeGuard":"def valid_gcs_path(p):\n    return isinstance(p, str) and p.startswith('gs://') and p.count('/') >= 3","tryCatchPattern":"try:\n    fs.rename(srcs, dsts)\nexcept BeamIOError as e:\n    for (src, dst), err in e.exception_details.items():\n        logging.error('rename %s -> %s failed: %s', src, dst, err)","preventionTips":["Avoid concurrent mutation of renamed objects","Ensure delete permission on source bucket and create on destination","Verify sources exist before rename","Handle copy and delete phase failures separately via exception_details"],"tags":["gcs","io","rename","aggregate-error"],"backgroundTag":"file-copy-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}