{"record":{"id":"b65b5a9fec520266","repo":"apache/beam","slug":"unable-to-rename-unequal-number-of-sources-and-destinations","errorCode":null,"errorMessage":"Unable to rename unequal number of sources and destinations","messagePattern":"Unable to rename unequal number of sources and destinations","errorType":"validation","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/aws/s3filesystem.py","lineNumber":215,"sourceCode":"      message = 'Unable to copy unequal number of sources and destinations'\n      raise BeamIOError(message)\n    src_dest_pairs = list(zip(source_file_names, destination_file_names))\n    return s3io.S3IO(options=self._options).copy_paths(src_dest_pairs)\n\n  def rename(self, source_file_names, destination_file_names):\n    \"\"\"Rename the files at the source list to the destination list.\n    Source and destination lists should be of the same size.\n\n    Args:\n      source_file_names: List of file paths that need to be moved\n      destination_file_names: List of destination_file_names for the files\n\n    Raises:\n      ``BeamIOError``: if any of the rename operations fail\n    \"\"\"\n    if not len(source_file_names) == len(destination_file_names):\n      message = 'Unable to rename unequal number of sources and destinations'\n      raise BeamIOError(message)\n    src_dest_pairs = list(zip(source_file_names, destination_file_names))\n    results = s3io.S3IO(options=self._options).rename_files(src_dest_pairs)\n    exceptions = {(src, dest): error\n                  for (src, dest, error) in results if error is not None}\n    if exceptions:\n      raise BeamIOError(\"Rename operation failed\", exceptions)\n\n  def exists(self, path):\n    \"\"\"Check if the provided path exists on the FileSystem.\n\n    Args:\n      path: string path that needs to be checked.\n\n    Returns: boolean flag indicating if path exists\n    \"\"\"\n    try:\n      return s3io.S3IO(options=self._options).exists(path)\n    except Exception as e:  # pylint: disable=broad-except","sourceCodeStart":197,"sourceCodeEnd":233,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/aws/s3filesystem.py#L197-L233","documentation":"Raised by S3FileSystem.rename when the source and destination path lists passed to BeamIO rename have different lengths. Beam's FileSystem.rename performs pairwise renames, so a mismatch means the operation cannot be mapped. The library fails fast with BeamIOError before touching S3.","triggerScenarios":"Calling S3FileSystem.rename(source_file_names, destination_file_names) with len(source_file_names) != len(destination_file_names), e.g. after filtering one list without the other.","commonSituations":"Bulk rename jobs built by collecting matching files then pairing them with a separately computed destination list; a filter on one list only; off-by-one slicing; paths deduplicated on one side.","solutions":["Verify len(source_file_names) == len(destination_file_names) before calling rename","Generate destinations by mapping over the source list (e.g. [src.replace(a,b) for src in sources]) instead of maintaining a separate list","If intentional, call rename per-pair for the subset that matches"],"exampleFix":"// before\nfs.rename(sorted_sources, destinations)  # lengths differ\n// after\nassert len(sorted_sources) == len(destinations)\nfs.rename(sorted_sources, destinations)","handlingStrategy":"validation","validationCode":"if len(sources) != len(destinations):\n    raise ValueError(f'length mismatch: {len(sources)} sources vs {len(destinations)} destinations')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive destination lists by mapping over sources, never maintain them separately","Assert equal lengths in tests for bulk-rename helpers"],"tags":["s3","io","argument-validation","beam"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}