{"record":{"id":"17525aacc15f53ff","repo":"apache/beam","slug":"you-cannot-delete-a-cluster-in-project","errorCode":null,"errorMessage":"You cannot delete a cluster in project: {}","messagePattern":"You cannot delete a cluster in project: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":403,"severity":"error","filePath":"sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py","lineNumber":250,"sourceCode":"\n  def cleanup(self) -> None:\n    \"\"\"Deletes the cluster that uses the attributes initialized\n    with the DataprocClusterManager instance.\"\"\"\n    try:\n      self._cluster_client.delete_cluster(\n          request={\n              'project_id': self.cluster_metadata.project_id,\n              'region': self.cluster_metadata.region,\n              'cluster_name': self.cluster_metadata.cluster_name,\n          })\n      self.cleanup_staging_files()\n    except Exception as e:\n      if e.code == 403:\n        _LOGGER.error(\n            'Due to insufficient project permissions, '\n            'unable to clean up the default cluster: %s',\n            self.cluster_metadata.cluster_name)\n        raise ValueError(\n            'You cannot delete a cluster in project: {}'.format(\n                self.cluster_metadata.project_id))\n      elif e.code == 404:\n        _LOGGER.error(\n            'Cluster does not exist: %s', self.cluster_metadata.cluster_name)\n        raise ValueError(\n            'Cluster was not found: {}'.format(\n                self.cluster_metadata.cluster_name))\n      else:\n        _LOGGER.error(\n            'Failed to delete cluster: %s', self.cluster_metadata.cluster_name)\n        raise e\n\n  def get_cluster_details(self) -> dataproc_v1.Cluster:\n    \"\"\"Gets the Dataproc_v1 Cluster object for the current cluster manager.\"\"\"\n    try:\n      return self._cluster_client.get_cluster(\n          request={","sourceCodeStart":232,"sourceCodeEnd":268,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py#L232-L268","documentation":"During cluster cleanup/delete, a Dataproc 403 (insufficient permissions) is re-raised as ValueError('You cannot delete a cluster in project: ...'), meaning the caller cannot delete clusters in that project.","triggerScenarios":"cleanup()/_cleanup invoked (e.g. notebook teardown or explicit delete) where the Dataproc delete API returns 403 due to missing dataproc.clusters.delete permission on the project.","commonSituations":"Notebook user's credentials differ from the creator's; viewer-only Dataproc role; service account restrictions or org policy preventing deletion.","solutions":["Grant roles/dataproc.editor or dataproc.clusters.delete permission to the caller in that project.","Delete the cluster with an account/service account that created it or has admin rights.","Verify project_id is correct; you may be deleting in a different project than intended.","As fallback, manually remove the orphaned cluster via console/gcloud with sufficient credentials."],"exampleFix":"// before\n# 403 on delete\nmanager.cleanup()\n// after\n# grant: gcloud projects add-iam-policy-binding PROJECT --member=... --role=roles/dataproc.editor\nmanager.cleanup()","handlingStrategy":"try-catch","validationCode":"# preflight permission: attempt a harmless dataproc call\nfrom google.cloud import dataproc_v1\nclient = dataproc_v1.ClusterControllerClient(client_options={'api_endpoint': f'{region}-dataproc.googleapis.com:443'})\nclusters = client.list_clusters(request={'project_id': project, 'region': region})  # raises early if no access","typeGuard":null,"tryCatchPattern":"try:\n    manager.cleanup()\nexcept ValueError as e:\n    if 'You cannot delete a cluster in project' in str(e):\n        escalate_credentials() or delete_manually_with_admin()  # avoid orphaned clusters\n    else:\n        raise","preventionTips":["Delete clusters with credentials that created them or hold roles/dataproc.editor.","Use a dedicated service account with consistent roles for create AND cleanup.","Confirm the target project id before cleanup to avoid cross-project 403s."],"tags":["apache-beam","dataproc","gcp","permissions","cluster"],"backgroundTag":"insufficient-permissions","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}