apache/beam · error · IOError

err (re-raised OSError during delete)

Error message

err (re-raised OSError during delete)

What it means

Inside _delete_path, when os.remove/shutil.rmtree raises OSError, the code re-raises it as IOError(err). This can be raised per-path during delete(); in the current code try_delete catches it, but this specific raise at line 347 surfaces when the underlying OS delete fails (permission denied, file locked, etc.).

Solutions

1. Check and fix file/directory permissions (chmod/chown) so the process can delete the target.
2. Ensure no other process holds the file open before deleting.
3. Retry the delete after a short delay if the failure is transient (e.g. NFS staleness).
4. Catch the error per-path and record it; delete() already collects per-path exceptions and reports them together.

Example fix

// before
fs.delete(['/tmp/locked.txt'])
// after
import time
for attempt in range(3):
    try:
        fs.delete(['/tmp/locked.txt'])
        break
    except IOError as e:
        time.sleep(1)  # retry transient deletion failure

import os
assert os.access(os.path.dirname(path) or '.', os.W_OK), 'cannot delete in dir'

try:
    fs.delete([path])
except BeamIOError as e:
    log.warning('delete failed for %s: %s', path, e)

When it happens

Trigger: Calling delete()/try_delete() on a path where the file is open by another process, the filesystem is read-only, permissions deny removal, or the file disappears between the isdir check and the remove call (TOCTOU race).

Common situations: Deleting temp files on shared/NFS mounts with permission issues; cleanup jobs racing with a running writer process; removing files owned by another user; Windows file-locking.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• err (re-raised OSError from os.rename)
• can't change to old working directory
• can't change to temp directory
• can't get current working directory
• Cannot pickle file as it cannot be read

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/935a528234572ef9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/localfilesystem.py:347

    """Deletes files or directories at the provided paths.
    Directories will be deleted recursively.

    Args:
      paths: list of paths that give the file objects to be deleted

    Raises:
      ``BeamIOError``: if any of the delete operations fail
    """
    def _delete_path(path):
      """Recursively delete the file or directory at the provided path.
      """
      try:
        if os.path.isdir(path):
          shutil.rmtree(path)
        else:
          os.remove(path)
      except OSError as err:
        raise IOError(err)

    exceptions = {}

    def try_delete(path):
      try:
        _delete_path(path)
      except Exception as e:  # pylint: disable=broad-except
        exceptions[path] = e

    for match_result in self.match(paths):
      metadata_list = match_result.metadata_list

      if not metadata_list:
        exceptions[match_result.pattern] = \
          IOError('No files found to delete under: %s' % match_result.pattern)

      for metadata in match_result.metadata_list:
        try_delete(metadata.path)

View on GitHub (pinned to 12126d8942)