apache/beam · error · ValueError

Unable to rename a directory.

Error message

Unable to rename a directory.

What it means

BlobStorageIO.rename_files() works by copying individual blobs and deleting the originals; it cannot move a 'directory' prefix as a whole. If either the source or destination path ends with '/', the method refuses with ValueError because directory-level rename must go through rename (copy_tree) instead.

Solutions

1. Use client.rename(src_dir, dest_dir) (or copy_tree) for directory renames
2. Strip trailing slashes / rename individual files when using rename_files
3. Filter src_dest_pairs to blob file paths before calling

Example fix

// before
client.rename_files([['container/logs/', 'container/logs-old/']])
// after
client.rename('container/logs', 'container/logs-old')  # tree-level rename

def renameable_pairs(pairs):
    return [(s.rstrip('/'), d.rstrip('/')) for s, d in pairs]
pairs = renameable_pairs(src_dest_pairs)  # no trailing slashes remain

def is_file_path(p):
    return not p.endswith('/')

try:
    client.rename_files(pairs)
except ValueError:
    client.rename(src_dir, dest_dir)  # tree-level fallback

When it happens

Trigger: Calling rename_files([['container/dir/', 'container/newdir/']]) or passing a dest with a trailing slash; batch jobs that normalize prefixes with trailing slashes.

Common situations: Confusing rename_files (file-level) with rename/copy_tree (tree-level); path-joining utilities that leave trailing slashes on prefixes.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Basepath %r must be an Azure Blob Storage path.
• Invalid file open mode
• Invalid path
• Path %r must be Azure Blob Storage path.
• Support for move options is not yet implemented.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/59a3c613f893eed9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/azure/blobstorageio.py:301

  # underlying copy and delete operations are already idempotent operations
  # protected by retry decorators.
  def rename_files(self, src_dest_pairs):
    """Renames the given Azure Blob Storage blobs from src to dest.

    Args:
      src_dest_pairs: List of (src, dest) tuples of
                      azfs://<storage-account>/<container>/[name]
                      file paths to rename from src to dest.
    Returns: List of tuples of (src, dest, exception) in the same order as the
             src_dest_pairs argument, where exception is None if the operation
             succeeded or the relevant exception if the operation failed.
    """
    if not src_dest_pairs:
      return []

    for src, dest in src_dest_pairs:
      if src.endswith('/') or dest.endswith('/'):
        raise ValueError('Unable to rename a directory.')

    # Results from copy operation.
    copy_results = self.copy_paths(src_dest_pairs)
    paths_to_delete = \
        [src for (src, _, error) in copy_results if error is None]
    # Results from delete operation.
    delete_results = self.delete_files(paths_to_delete)

    # Get rename file results (list of tuples).
    results = []

    # Using a dictionary will make the operation faster.
    delete_results_dict = {src: error for (src, error) in delete_results}

    for src, dest, error in copy_results:
      # If there was an error in the copy operation.
      if error is not None:
        results.append((src, dest, error))

View on GitHub (pinned to 12126d8942)