apache/beam · error · ValueError

Invalid file open mode

Error message

Invalid file open mode: %s.

What it means

BlobStorageIO.open() only supports read ('r'/'rb') and write ('w'/'wb') modes for Azure Blob Storage. Any other mode string (e.g. 'a', 'x', 'r+') reaches the final else branch and raises ValueError. The library deliberately restricts modes because Azure blob access is mapped to either a downloader or an uploader stream.

Solutions

1. Use mode 'r' or 'rb' for reading and 'w' or 'wb' for writing only
2. For append semantics, read the blob fully, then rewrite it in 'w' mode with concatenated content
3. Validate/normalize the mode string before calling open()

Example fix

// before
with client.open('gs-like/path/blob.txt', 'a') as f:
    f.write('more')
// after
existing = client.open('path/blob.txt', 'r').read() if client.exists('path/blob.txt') else ''
with client.open('path/blob.txt', 'w') as f:
    f.write(existing + 'more')

def valid_mode(mode):
    return mode in ('r', 'rb', 'w', 'wb')
if not valid_mode(mode):
    raise ValueError(f'mode must be r/rb/w/wb, got {mode}')

def is_supported_mode(mode):
    return isinstance(mode, str) and mode in {'r', 'rb', 'w', 'wb'}

try:
    f = client.open(path, mode)
except ValueError:
    logging.error('unsupported open mode %r for Azure blob', mode)
    raise

When it happens

Trigger: Calling client.open(path, mode='a'), mode='x', mode='ab', or any mode with '+' (read-write) on an azure.BlobStorageIO client; also passing a mode from misconfigured pipeline options.

Common situations: Porting code written for local files or GCS that appends to logs ('a'); typo in mode string like 'w+'; generic file-helper utilities that pass through arbitrary modes.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Basepath %r must be an Azure Blob Storage path.
• Invalid path
• Path %r must be Azure Blob Storage path.
• Unable to rename a directory.
• Can't read parse private key

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7f2df9658417178b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/azure/blobstorageio.py:158

    Returns:
      Azure Blob Storage file object.
    Raises:
      ValueError: Invalid open file mode.
    """
    if mode == 'r' or mode == 'rb':
      downloader = BlobStorageDownloader(
          self.client, filename, buffer_size=read_buffer_size)
      return io.BufferedReader(
          DownloaderStream(
              downloader, read_buffer_size=read_buffer_size, mode=mode),
          buffer_size=read_buffer_size)
    elif mode == 'w' or mode == 'wb':
      uploader = BlobStorageUploader(self.client, filename, mime_type)
      return io.BufferedWriter(
          UploaderStream(uploader, mode=mode), buffer_size=128 * 1024)
    else:
      raise ValueError('Invalid file open mode: %s.' % mode)

  @retry.with_exponential_backoff(
      retry_filter=retry.retry_on_beam_io_error_filter)
  def copy(self, src, dest):
    """Copies a single Azure Blob Storage blob from src to dest.

    Args:
      src: Blob Storage file path pattern in the form
           azfs://<storage-account>/<container>/[name].
      dest: Blob Storage file path pattern in the form
            azfs://<storage-account>/<container>/[name].

    Raises:
      TimeoutError: on timeout.
    """
    src_storage_account, src_container, src_blob = parse_azfs_path(
        src, get_account=True)
    dest_container, dest_blob = parse_azfs_path(dest)

View on GitHub (pinned to 12126d8942)