apache/beam · error · BeamIOError

Unable to rename unequal number of sources and destinations.

Error message

Unable to rename unequal number of sources and destinations.

What it means

apache_beam.io.azure.blobstoragefilesystem.BlobStorageFileSystem.rename() raises BeamIOError with this message when the source_file_names and destination_file_names lists passed to it do not have the same number of elements. The bulk rename API works on (source, destination) pairs, so an unequal count would leave some sources unmapped or some destinations undefined; the check happens before any rename is attempted, so no files are moved.

Source

Thrown at sdks/python/apache_beam/io/azure/blobstoragefilesystem.py:211

      message = 'Unable to copy unequal number of sources and destinations.'
      raise BeamIOError(message)
    src_dest_pairs = list(zip(source_file_names, destination_file_names))
    return self._blobstorageIO().copy_paths(src_dest_pairs)

  def rename(self, source_file_names, destination_file_names):
    """Rename the files at the source list to the destination list.
    Source and destination lists should be of the same size.

    Args:
      source_file_names: List of file paths that need to be moved
      destination_file_names: List of destination_file_names for the files

    Raises:
      ``BeamIOError``: if any of the rename operations fail
    """
    if not len(source_file_names) == len(destination_file_names):
      message = 'Unable to rename unequal number of sources and destinations.'
      raise BeamIOError(message)
    src_dest_pairs = list(zip(source_file_names, destination_file_names))
    results = self._blobstorageIO().rename_files(src_dest_pairs)
    # Retrieve exceptions.
    exceptions = {(src, dest): error
                  for (src, dest, error) in results if error is not None}
    if exceptions:
      raise BeamIOError("Rename operation failed.", exceptions)

  def exists(self, path):
    """Check if the provided path exists on the FileSystem.

    Args:
      path: string path that needs to be checked.

    Returns: boolean flag indicating if path exists
    """
    try:
      return self._blobstorageIO().exists(path)

View on GitHub (pinned to 12126d8942)

Solutions

Verify len(source_file_names) == len(destination_file_names) before calling rename and fix the code that builds the shorter list
If you intended a one-to-one rename of one file, pass single-element lists, not a bare string wrapped inconsistently
Log both lists (or their lengths) at the point of call to find where they diverge
If sources were discovered dynamically, derive destinations from the same iteration so they stay paired

Example fix

// before
fs.rename(sources, destinations)  # lengths differ
// after
assert len(sources) == len(destinations), f"{len(sources)} sources vs {len(destinations)} destinations"
fs.rename(sources, destinations)

Defensive patterns

Strategy: validation

Validate before calling

if len(source_file_names) != len(destination_file_names):
    raise ValueError(
        f"rename: {len(source_file_names)} sources != {len(destination_file_names)} destinations")

Prevention

Build source and destination lists in the same loop/zip so they cannot diverge
Add an assert on equal lengths at the call site during development
Derive destinations from sources programmatically (e.g. prefix replacement) instead of hand-writing them
Write a unit test that feeds mismatched lists to catch list-building bugs

When it happens

Trigger: Calling rename(source_file_names, destination_file_names) where len(source_file_names) != len(destination_file_names), e.g. building the lists in separate loops where one silently drops items, or appending destinations only conditionally.

Common situations: Pipeline code that computes source file lists from a glob match but hand-writes or partially generates the destination list; a bug where a filtered-out source still has its destination appended (or vice versa); typos like passing the same list twice for a different operation.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/14b43bd1af7c326e. Report an issue: GitHub.