{"record":{"id":"06eaa38c8f544334","repo":"apache/beam","slug":"delete-operation-failed-blobstoragefilesystem","errorCode":null,"errorMessage":"Delete operation failed","messagePattern":"Delete operation failed","errorType":"exception","errorClass":"BeamIOError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/azure/blobstoragefilesystem.py","lineNumber":319,"sourceCode":"  def delete(self, paths):\n    \"\"\"Deletes files or directories at the provided paths.\n    Directories will be deleted recursively.\n\n    Args:\n      paths: list of paths that give the file objects to be deleted\n\n    Raises:\n      ``BeamIOError``: if any of the delete operations fail\n    \"\"\"\n    results = self._blobstorageIO().delete_paths(paths)\n    # Retrieve exceptions.\n    exceptions = {\n        path: error\n        for (path, error) in results.items() if error is not None\n    }\n\n    if exceptions:\n      raise BeamIOError(\"Delete operation failed\", exceptions)\n\n  def report_lineage(self, path, lineage):\n    try:\n      components = blobstorageio.parse_azfs_path(\n          path, blob_optional=True, get_account=True)\n    except ValueError:\n      # report lineage is fail-safe\n      traceback.print_exc()\n      return\n    if components and not components[-1]:\n      components = components[:-1]\n    lineage.add('abs', *components, last_segment_sep='/')\n","sourceCodeStart":301,"sourceCodeEnd":332,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/azure/blobstoragefilesystem.py#L301-L332","documentation":"BlobStorageFileSystem.delete(paths) deletes files/directories recursively via BlobStorageIO.delete_files() and raises BeamIOError('Delete operation failed', exceptions) when any path failed, where exceptions maps each failed path to its error. Unlike rename, there is no count check — this error only reflects remote per-path failures.","triggerScenarios":"Calling fs.delete(paths) where at least one path fails to delete: blob/container does not exist, credentials lack delete permission, container has a delete retention policy/lease, or transient Azure errors. Results are collected per path and any non-None error triggers the raise.","commonSituations":"Cleanup of temp/staging blobs that were already removed by a lifecycle management policy or another worker; service principal missing 'Storage Blob Data Contributor' delete rights; deleting containers with immutable/soft-delete policies; throttling during large batch deletes.","solutions":["Inspect BeamIOError.exception_details to see which paths failed and retry only those","Ensure credentials/SAS have delete permissions (Storage Blob Data Contributor or equivalent)","Disable or account for soft-delete/retention/lease policies on the container","Make deletes idempotent: treat 'not found' failures as success in cleanup code by filtering them from a retry"],"exampleFix":"// before\nfs.delete(paths)\n// after\ntry:\n    fs.delete(paths)\nexcept BeamIOError as e:\n    retry = [p for p, err in e.exception_details.items() if not fs.exists(p) is False or True]\n    # simpler: retry paths that still exist\n    retry = [p for p in e.exception_details if fs.exists(p)]\n    if retry:\n        fs.delete(retry)","handlingStrategy":"try-catch","validationCode":"# filter paths that still exist and are safe to delete\nexisting = [p for p in paths if fs.exists(p)]","typeGuard":null,"tryCatchPattern":"try:\n    fs.delete(paths)\nexcept BeamIOError as e:\n    failed = e.exception_details  # dict {path: error}\n    retry = [p for p in failed if fs.exists(p)]  # skip already-deleted\n    if retry:\n        fs.delete(retry)","preventionTips":["Make deletes idempotent — ignore 'not found' failures in cleanup jobs","Grant delete permissions (Storage Blob Data Contributor) to the identity used","Account for soft-delete/retention policies that block hard deletes","Batch large deletions to avoid throttling"],"tags":["python","apache-beam","azure-blob-storage","filesystem-io","bulk-operation"],"backgroundTag":"file-delete-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}