{"record":{"id":"74ff934316c49c7a","repo":"apache/beam","slug":"metadata-operation-failed-gcsfilesystem","errorCode":null,"errorMessage":"Metadata operation failed","messagePattern":"Metadata operation failed","errorType":"exception","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/gcsfilesystem.py","lineNumber":343,"sourceCode":"      raise BeamIOError(\"Checksum operation failed\", {path: e})\n\n  def metadata(self, path):\n    \"\"\"Fetch metadata fields of a file on the FileSystem.\n\n    Args:\n      path: string path of a file.\n\n    Returns:\n      :class:`~apache_beam.io.filesystem.FileMetadata`.\n\n    Raises:\n      ``BeamIOError``: if path isn't a file or doesn't exist.\n    \"\"\"\n    try:\n      file_metadata = self._gcsIO()._status(path)\n      return FileMetadata(path, file_metadata['size'], file_metadata['updated'])\n    except Exception as e:  # pylint: disable=broad-except\n      raise BeamIOError(\"Metadata operation failed\", {path: e})\n\n  def delete(self, paths):\n    \"\"\"Deletes files or directories at the provided paths.\n    Directories will be deleted recursively.\n\n    Args:\n      paths: list of paths that give the file objects to be deleted\n    \"\"\"\n\n    exceptions = {}\n\n    for path in paths:\n      if path.endswith('/'):\n        self._gcsIO().delete(path, recursive=True)\n        continue\n      else:\n        path_to_use = path\n      match_result = self.match([path_to_use])[0]","sourceCodeStart":325,"sourceCodeEnd":361,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/gcsfilesystem.py#L325-L361","documentation":"GcsFileSystem.metadata raises BeamIOError('Metadata operation failed') when fetching file metadata via gcsio._status(path) throws. The original exception is preserved in exception_details keyed by the path.","triggerScenarios":"Calling GcsFileSystem.metadata(path) where the path is malformed, the object does not exist, or the GCS API call fails (permissions, network).","commonSituations":"Querying size/updated of an object that was never written or already deleted; wrong bucket name; path missing gs:// scheme.","solutions":["Validate the path format and existence (parse_gcs_path / exists) before calling metadata.","Inspect exception_details for the underlying NotFound/permission error.","Fix the gs:// path or point to an existing object.","Catch BeamIOError to handle missing objects gracefully."],"exampleFix":"// before\nmeta = fs.metadata(path)\n// after\ntry:\n  meta = fs.metadata(path)\nexcept BeamIOError:\n  meta = None  # object missing or unreadable","handlingStrategy":"validation","validationCode":"if not fs.exists(path):\n    raise FileNotFoundError(path)","typeGuard":"def valid_gcs_path(p):\n    return isinstance(p, str) and p.startswith('gs://') and p.count('/') >= 3","tryCatchPattern":"try:\n    meta = fs.metadata(path)\nexcept BeamIOError as e:\n    logging.warning('metadata failed: %s', e.exception_details.get(path))\n    meta = None","preventionTips":["Verify object exists before fetching metadata","Validate gs:// path format","Handle missing objects gracefully in pipelines"],"tags":["gcs","metadata","io"],"backgroundTag":"file-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}