{"record":{"id":"a79bb3354e4fca05","repo":"apache/beam","slug":"delete-operation-failed-gcsfilesystem","errorCode":null,"errorMessage":"Delete operation failed","messagePattern":"Delete operation failed","errorType":"exception","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/gcsfilesystem.py","lineNumber":369,"sourceCode":"    \"\"\"\n\n    exceptions = {}\n\n    for path in paths:\n      if path.endswith('/'):\n        self._gcsIO().delete(path, recursive=True)\n        continue\n      else:\n        path_to_use = path\n      match_result = self.match([path_to_use])[0]\n      statuses = self._gcsIO().delete_batch(\n          [m.path for m in match_result.metadata_list])\n      for target, exception in statuses:\n        if exception:\n          exceptions[target] = exception\n\n    if exceptions:\n      raise BeamIOError(\"Delete operation failed\", exceptions)\n\n  def report_lineage(self, path, lineage):\n    try:\n      components = gcsio.parse_gcs_path(path, object_optional=True)\n    except ValueError:\n      # report lineage is fail-safe\n      traceback.print_exc()\n      return\n    if components and not components[-1]:\n      components = components[:-1]\n    lineage.add('gcs', *components, last_segment_sep='/')\n","sourceCodeStart":351,"sourceCodeEnd":381,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/gcsfilesystem.py#L351-L381","documentation":"GcsFileSystem.delete raises BeamIOError('Delete operation failed') when any of the requested deletions fails. It matches paths, then deletes each; per-target exceptions are aggregated into a single BeamIOError keyed by path.","triggerScenarios":"Calling GcsFileSystem.delete(paths) where at least one path cannot be deleted: object missing, permission denied on the bucket, or a matched target fails during deletion.","commonSituations":"Cleaning up temp files already removed by another worker; service account lacking storage.objects.delete; deleting prefixes with thousands of objects where some fail.","solutions":["Read exception_details to identify which paths failed and why.","Grant the service account storage.objects.delete permission on the bucket.","Check existence (exists/match) and skip already-deleted paths.","Retry the failed subset from exception_details."],"exampleFix":"// before\nfs.delete(paths)\n// after\ntry:\n  fs.delete(paths)\nexcept BeamIOError as e:\n  for target, err in e.exception_details.items():\n    logging.warning('failed to delete %s: %s', target, err)","handlingStrategy":"try-catch","validationCode":"targets = [m.path for m in fs.match([p if '*' in p else p + '*'])[0].metadata_list]\n","typeGuard":"def valid_gcs_path(p):\n    return isinstance(p, str) and p.startswith('gs://') and p.count('/') >= 3","tryCatchPattern":"try:\n    fs.delete(paths)\nexcept BeamIOError as e:\n    for target, err in e.exception_details.items():\n        logging.warning('delete %s failed: %s', target, err)","preventionTips":["Grant storage.objects.delete to the service account","Skip paths already deleted (idempotent cleanup)","Retry failed subset from exception_details","Avoid deleting files concurrently used by other workers"],"tags":["gcs","delete","io","aggregate-error"],"backgroundTag":"file-delete-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}