apache/beam · error · BeamIOError

Last updated operation failed

Error message

Last updated operation failed

What it means

BlobStorageFileSystem.last_updated() returns the UNIX epoch modification time via BlobStorageIO.last_updated() and re-raises any exception as BeamIOError('Last updated operation failed', {path: e}). As with size(), a nonexistent path is one of the causes covered by this error.

Source

Thrown at sdks/python/apache_beam/io/azure/blobstoragefilesystem.py:263

      return self._blobstorageIO().size(path)
    except Exception as e:  # pylint: disable=broad-except
      raise BeamIOError("Size operation failed", {path: e})

  def last_updated(self, path):
    """Get UNIX Epoch time in seconds on the FileSystem.

    Args:
      path: string path of file.

    Returns: float UNIX Epoch time

    Raises:
      ``BeamIOError``: if path doesn't exist.
    """
    try:
      return self._blobstorageIO().last_updated(path)
    except Exception as e:  # pylint: disable=broad-except
      raise BeamIOError("Last updated operation failed", {path: e})

  def checksum(self, path):
    """Fetch checksum metadata of a file on the
    :class:`~apache_beam.io.filesystem.FileSystem`.

    Args:
      path: string path of a file.

    Returns: string containing checksum

    Raises:
      ``BeamIOError``: if path isn't a file or doesn't exist.
    """
    try:
      return self._blobstorageIO().checksum(path)
    except Exception as e:  # pylint: disable=broad-except
      raise BeamIOError("Checksum operation failed", {path, e})

View on GitHub (pinned to 12126d8942)

Solutions

  1. Check fs.exists(path) before reading timestamps
  2. Inspect BeamIOError.exception_details[path] for the underlying Azure exception
  3. Validate the azfs:// path format and container name
  4. Refresh credentials / SAS tokens if the error is authentication-related

Example fix

// before
mtime = fs.last_updated(path)
// after
try:
    mtime = fs.last_updated(path)
except BeamIOError as e:
    logging.warning(f"last_updated failed for {path}: {e.exception_details}")
    mtime = 0  # treat as never-updated or re-raise as appropriate
Defensive patterns

Strategy: try-catch

Validate before calling

if not fs.exists(path):
    raise FileNotFoundError(f"no such blob: {path}")

Try / catch

try:
    mtime = fs.last_updated(path)
except BeamIOError as e:
    logging.error("last_updated failed for %s: %s", path, e.exception_details)
    mtime = 0  # or re-raise depending on watermark semantics

Prevention

When it happens

Trigger: Calling fs.last_updated(path) on a missing blob or when the Azure SDK raises (auth failure, network error, insufficient permissions).

Common situations: Incremental pipeline logic that compares file timestamps against a watermark but the upstream file has not landed yet; timezone/copy mistakes producing wrong container paths; expired credentials.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/29891675c731c534. Report an issue: GitHub.