apache/beam · error · BeamIOError

Exists operation failed

Error message

Exists operation failed

What it means

BlobStorageFileSystem.exists() wraps the underlying BlobStorageIO.exists() call and re-raises any exception as BeamIOError('Exists operation failed', {path: e}). This signals that existence could not be determined (e.g. network or auth failure), not that the file is missing — a missing file simply returns False.

Source

Thrown at sdks/python/apache_beam/io/azure/blobstoragefilesystem.py:231

    results = self._blobstorageIO().rename_files(src_dest_pairs)
    # Retrieve exceptions.
    exceptions = {(src, dest): error
                  for (src, dest, error) in results if error is not None}
    if exceptions:
      raise BeamIOError("Rename operation failed.", exceptions)

  def exists(self, path):
    """Check if the provided path exists on the FileSystem.

    Args:
      path: string path that needs to be checked.

    Returns: boolean flag indicating if path exists
    """
    try:
      return self._blobstorageIO().exists(path)
    except Exception as e:  # pylint: disable=broad-except
      raise BeamIOError("Exists operation failed", {path: e})

  def size(self, path):
    """Get size in bytes of a file on the FileSystem.

    Args:
      path: string filepath of file.

    Returns: int size of file according to the FileSystem.

    Raises:
      ``BeamIOError``: if path doesn't exist.
    """
    try:
      return self._blobstorageIO().size(path)
    except Exception as e:  # pylint: disable=broad-except
      raise BeamIOError("Size operation failed", {path: e})

  def last_updated(self, path):

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the wrapped exception in BeamIOError.exception_details[path] to identify the root cause (auth, network, etc.)
2. Validate Azure options (account name/key, connection string, endpoint) before calling
3. Test connectivity and credentials with a simple azure-storage-blob call or az CLI
4. Distinguish 'not found' (returns False) from this error — only the exception indicates an infrastructure problem

Example fix

// before
exists = fs.exists(path)
// after
try:
    exists = fs.exists(path)
except BeamIOError as e:
    logging.error(f"Could not check existence of {path}: {e.exception_details}")
    exists = False  # or re-raise, depending on semantics needed

# validate credentials/options before the call
from apache_beam.io.azure import blobstorageio
blobstorageio.parse_azfs_path(path)  # raises ValueError on malformed path

try:
    exists = fs.exists(path)
except BeamIOError as e:
    logging.error("exists check failed for %s: %s", path, e.exception_details)
    exists = False  # or re-raise; False here means 'unknown', not 'missing'

When it happens

Trigger: Calling fs.exists(path) with an azfs:// path when the Azure SDK raises: invalid/mismatched account credentials, network failure, malformed connection options, or throttling from the storage service.

Common situations: Checking file existence during pipeline setup with expired or misconfigured Azure credentials; connectivity issues in CI containers without network access to the storage endpoint; account name typo causing DNS/auth failures.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Size operation failed
• Last updated operation failed
• Checksum operation failed
• Metadata operation failed
• Unable to rename unequal number of sources and destinations.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3006d33c590ab8c1. Report an issue: GitHub.