apache/beam · error · IOError

ENOENT

ENOENT

Error message

Not found: %s

What it means

S3Downloader.__init__ fetches the object's metadata (HEAD request) when opening an S3 object for reading. If S3 returns HTTP 404, it translates this into IOError(errno.ENOENT, 'Not found: <path>'), the Python idiom for a missing file.

Source

Thrown at sdks/python/apache_beam/io/aws/s3io.py:562

class S3Downloader(Downloader):
  def __init__(self, client, path, buffer_size):
    self._client = client
    self._path = path
    self._bucket, self._name = parse_s3_path(path)
    self._buffer_size = buffer_size

    # Get object state.
    self._get_request = (
        messages.GetRequest(bucket=self._bucket, object=self._name))

    try:
      metadata = self._get_object_metadata(self._get_request)

    except messages.S3ClientError as e:
      if e.code == 404:
        raise IOError(errno.ENOENT, 'Not found: %s' % self._path)
      else:
        logging.error('HTTP error while requesting file %s: %s', self._path, 3)
        raise

    self._size = metadata.size

  @retry.with_exponential_backoff(
      retry_filter=retry.retry_on_server_errors_and_timeout_filter)
  def _get_object_metadata(self, get_request):
    return self._client.get_object_metadata(get_request)

  @property
  def size(self):
    return self._size

  def get_range(self, start, end):
    return self._client.get_range(self._get_request, start, end)

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the key exists (aws s3 ls s3://bucket/key or client.head_object) before opening.
2. Check the bucket name, region and key spelling/case.
3. Catch the IOError and handle ENOENT explicitly where absence is expected.
4. Confirm credentials grant s3:GetObject / s3:ListBucket (hidden 404s can result from permission filtering).

Example fix

// before
with s3io.open('s3://%s/%s' % (bucket, key)) as f: ...
// after
if not io.exists(path):
    return None
try:
    with io.open(path) as f: ...
except IOError as e:
    if e.errno != errno.ENOENT: raise

def s3_key_exists(io, path):
    try:
        return len(io.list_files(path)) > 0
    except Exception:
        return False

def is_missing_file(exc):
    return isinstance(exc, IOError) and exc.errno == errno.ENOENT

try:
    with io.open(path) as f:
        data = f.read()
except IOError as e:
    if e.errno == errno.ENOENT:
        data = None  # handle absence
    else:
        raise

When it happens

Trigger: Opening a nonexistent or already-deleted key via S3IO.open('s3://bucket/missing', 'r'); a read path constructed from a typo or from a listing that has since changed; wrong bucket/region so the key does not resolve.

Common situations: Race with a deletion job between listing and reading; wrong bucket name or AWS credentials scoped to another account; case-sensitivity mistakes in keys (S3 keys are case-sensitive).

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Basepath %r must be S3 path.
• Path %r must be S3 path.
• Invalid path: %s
• List operation failed
• Rename operation failed

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/44859da9fba86094. Report an issue: GitHub.