apache/beam · error · BeamIOError

Size operation failed

Error message

Size operation failed

What it means

LocalFileSystem.size returns the size of a single file via os.path.getsize, and wraps any failure (missing path, path is a directory, permission error) in BeamIOError('Size operation failed', {path: original_error}). The docstring documents BeamIOError as the failure mode when the path doesn't exist. Note this method only handles single paths, not globs.

Solutions

1. Check existence first with FileSystems.exists(path) or os.path.exists before calling size.
2. Verify the path is a file, not a directory, for a file size.
3. Catch BeamIOError and read exception_details[path] for the root cause.
4. Use FileSystems.match([glob]) and read metadata_list[*].size_in_bytes when you need sizes for globs or possibly-missing files.
5. Add retry with backoff if the file may be transiently absent due to concurrent writes.

Example fix

// before
size = fs.size(path)  # BeamIOError if file missing
// after
if fs.exists(path):
    size = fs.size(path)
else:
    size = None

import os
if os.path.isfile(path):
    size = os.path.getsize(path)
else:
    size = None

from apache_beam.io.filesystem import BeamIOError
try:
    size = fs.size(path)
except BeamIOError as e:
    size = None  # or log e.exception_details[path]

When it happens

Trigger: Calling LocalFileSystem.size(path) (or FileSystems.size via matching filesystem) where os.path.getsize fails: the file does not exist, path points to a directory (getsize works but semantics differ per platform; error mainly for missing/inaccessible paths), or permission is denied.

Common situations: Querying the size of a not-yet-created output file; race between another process deleting the file and the size call; passing a directory path where a file path is required.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Copy operation failed
• err (re-raised OSError during copy)
• err (re-raised OSError from os.makedirs)
• err (re-raised OSError from os.rename)
• Field ' ' not found in input schema fields

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5eed6587d06aace9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/localfilesystem.py:279

    Returns: boolean flag indicating if path exists
    """
    return os.path.exists(path)

  def size(self, path):
    """Get size of path on the FileSystem.

    Args:
      path: string path in question.

    Returns: int size of path according to the FileSystem.

    Raises:
      ``BeamIOError``: if path doesn't exist.
    """
    try:
      return os.path.getsize(path)
    except Exception as e:  # pylint: disable=broad-except
      raise BeamIOError("Size operation failed", {path: e})

  def last_updated(self, path):
    """Get UNIX Epoch time in seconds on the FileSystem.

    Args:
      path: string path of file.

    Returns: float UNIX Epoch time

    Raises:
      ``BeamIOError``: if path doesn't exist.
    """
    if not self.exists(path):
      raise BeamIOError('Path does not exist: %s' % path)
    return os.path.getmtime(path)

  def checksum(self, path):
    """Fetch checksum metadata of a file on the

View on GitHub (pinned to 12126d8942)