apache/beam · error · BeamIOError

Path does not exist

Error message

Path does not exist: %s

What it means

LocalFileSystem.last_updated returns the modification time of a file, but first checks self.exists(path) and raises BeamIOError('Path does not exist: %s' % path) when the check fails. It exists precisely to give a clear error instead of a raw OSError from os.path.getmtime when the path is missing or inaccessible.

Solutions

1. Verify the path exists and is spelled correctly (absolute vs relative, URL scheme) before calling last_updated.
2. Check FileSystems.exists(path) in your own code first if you want to handle absence gracefully.
3. Catch BeamIOError and treat a missing path as 'no timestamp available' in your workflow.
4. If the file should exist, investigate why it is missing: deleted by retention/cleanup jobs, wrong stage/output directory, or failed upstream step.
5. Use checksum/metadata variants only when you actually need them; ensure path is a file, since exists may hold for directories too.

Example fix

// before
mtime = fs.last_updated(path)
// after
from apache_beam.io.filesystem import BeamIOError
try:
    mtime = fs.last_updated(path)
except BeamIOError:
    mtime = None  # path absent or inaccessible

from apache_beam.io.filesystems import FileSystems
if not FileSystems.exists(path):
    raise FileNotFoundError(path)

from apache_beam.io.filesystem import BeamIOError
try:
    mtime = fs.last_updated(path)
except BeamIOError:
    mtime = None  # treat as 'no timestamp available'

When it happens

Trigger: Calling LocalFileSystem.last_updated(path) for a path that does not exist, has been deleted, is a broken symlink, or is not accessible so that exists() returns False. Also triggered by transient races where the file is removed between the exists check and getmtime.

Common situations: Reading mtime of an output file before the pipeline produced it; checking files on a mount that is temporarily unavailable; stale cached paths from a previous run.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Copy operation failed
• err (re-raised OSError during copy)
• err (re-raised OSError from os.makedirs)
• err (re-raised OSError from os.rename)
• Field ' ' not found in input schema fields

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bbde3a48840fa307. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/localfilesystem.py:293

    """
    try:
      return os.path.getsize(path)
    except Exception as e:  # pylint: disable=broad-except
      raise BeamIOError("Size operation failed", {path: e})

  def last_updated(self, path):
    """Get UNIX Epoch time in seconds on the FileSystem.

    Args:
      path: string path of file.

    Returns: float UNIX Epoch time

    Raises:
      ``BeamIOError``: if path doesn't exist.
    """
    if not self.exists(path):
      raise BeamIOError('Path does not exist: %s' % path)
    return os.path.getmtime(path)

  def checksum(self, path):
    """Fetch checksum metadata of a file on the
    :class:`~apache_beam.io.filesystem.FileSystem`.

    Args:
      path: string path of a file.

    Returns: string containing file size.

    Raises:
      ``BeamIOError``: if path isn't a file or doesn't exist.
    """
    if not self.exists(path):
      raise BeamIOError('Path does not exist: %s' % path)
    return str(os.path.getsize(path))

View on GitHub (pinned to 12126d8942)