apache/beam · error · NotFound

Object not found

Error message

Object %s not found

What it means

GcsIO._gcs_object raises google.cloud.exceptions.NotFound ('Object %s not found') when bucket.get_blob returns None, i.e. the object does not exist. This helper backs checksum, size, kms_key, last_updated, and _status, so all of those surface NotFound for missing objects.

Solutions

1. Check existence first (gcsio.exists(path)) before querying metadata.
2. Correct the gs:// path, including exact object-name casing.
3. Catch NotFound and return a sensible default (e.g. None / -1).
4. Investigate concurrent deletions or retention/lifecycle rules if the object should exist.

Example fix

// before
size = gcsio.size(path)
// after
size = gcsio.size(path) if gcsio.exists(path) else None

if not gcsio.exists(path):
    raise FileNotFoundError(path)

def is_gcs_path(p):
    return isinstance(p, str) and re.match(r'^gs://[^/]+/.+$', p) is not None

from google.cloud.exceptions import NotFound
try:
    size = gcsio.size(path)
except NotFound:
    size = None

When it happens

Trigger: Calling any of checksum/size/kms_key/last_updated/_status with a gs:// path whose object doesn't exist, has a typo, or was deleted before the call.

Common situations: Stat-ing an output object before the pipeline wrote it; lifecycle rules deleting objects between write and read; case-sensitive object name typos.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• error getting last modified time for
• Metadata operation failed
• Rewrite from to has failed. Either source or sink not…
• source blob not found during copying
• All workers communicate through gRPC should have worker_id…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8d1606ffa1d766a3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/gcsio.py:619

    if hasattr(gcs_object, 'size'):
      file_status['size'] = gcs_object.size
    return file_status

  def _gcs_object(self, path):
    """Returns a gcs object for the given path

    This method does not perform glob expansion. Hence the given path must be
    for a single GCS object. The method will make HTTP requests.

    Returns: GCS object.
    """
    bucket_name, blob_name = parse_gcs_path(path)
    bucket = self.client.bucket(bucket_name)
    blob = bucket.get_blob(blob_name, retry=self._storage_client_retry)
    if blob:
      return blob
    else:
      raise NotFound('Object %s not found', path)

  def list_files(self, path, with_metadata=False):
    """Lists files matching the prefix.

    Args:
      path: GCS file path pattern in the form gs://<bucket>/[name].
      with_metadata: Experimental. Specify whether returns file metadata.

    Returns:
      If ``with_metadata`` is False: generator of tuple(file name, size); if
      ``with_metadata`` is True: generator of
      tuple(file name, tuple(size, timestamp)).
    """
    bucket_name, prefix = parse_gcs_path(path, object_optional=True)
    file_info = set()
    counter = 0
    start_time = time.time()
    if with_metadata:

View on GitHub (pinned to 12126d8942)