{"record":{"id":"78be16d77ff4d315","repo":"apache/beam","slug":"you-cannot-view-clusters-in-project","errorCode":null,"errorMessage":"You cannot view clusters in project: {}","messagePattern":"You cannot view clusters in project: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":403,"severity":"error","filePath":"sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py","lineNumber":279,"sourceCode":"            'Failed to delete cluster: %s', self.cluster_metadata.cluster_name)\n        raise e\n\n  def get_cluster_details(self) -> dataproc_v1.Cluster:\n    \"\"\"Gets the Dataproc_v1 Cluster object for the current cluster manager.\"\"\"\n    try:\n      return self._cluster_client.get_cluster(\n          request={\n              'project_id': self.cluster_metadata.project_id,\n              'region': self.cluster_metadata.region,\n              'cluster_name': self.cluster_metadata.cluster_name\n          })\n    except Exception as e:\n      if e.code == 403:\n        _LOGGER.error(\n            'Due to insufficient project permissions, '\n            'unable to retrieve information for cluster: %s',\n            self.cluster_metadata.cluster_name)\n        raise ValueError(\n            'You cannot view clusters in project: {}'.format(\n                self.cluster_metadata.project_id))\n      elif e.code == 404:\n        _LOGGER.error(\n            'Cluster does not exist: %s', self.cluster_metadata.cluster_name)\n        raise ValueError(\n            'Cluster was not found: {}'.format(\n                self.cluster_metadata.cluster_name))\n      else:\n        _LOGGER.error(\n            'Failed to get information for cluster: %s',\n            self.cluster_metadata.cluster_name)\n        raise e\n\n  def wait_for_cluster_to_provision(self) -> None:\n    while self.get_cluster_details().status.state.name == 'CREATING':\n      time.sleep(15)\n","sourceCodeStart":261,"sourceCodeEnd":297,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py#L261-L297","documentation":"Raised by ClustersManager.get_cluster_details() when the Dataproc API returns HTTP 403 for a cluster info request. Beam wraps this in a ValueError indicating the credentials in use lack permission to view clusters in the given project. Callers include wait_for_cluster_to_provision, get_staging_location and parse_master_url_and_dashboard, so provisioning can also fail with this.","triggerScenarios":"Calling get_cluster_details() (or any caller: wait_for_cluster_to_provision, get_staging_location, parse_master_url_and_dashboard) when the authenticated account lacks dataproc.clusters.get on the project, or the Dataproc API is not enabled / service account lacks the Dataproc Viewer role.","commonSituations":"Using a default service account without Dataproc roles; running in a notebook on a machine with restricted ADC credentials; organization policy blocking access; wrong project_id in ClusterMetadata pointing at a project the user cannot see.","solutions":["Grant the account/dataproc service account the roles/dataproc.editor (or at least roles/dataproc.viewer) IAM role on the project.","Verify the correct project_id in cluster_metadata; you may be querying the wrong project.","Re-authenticate with sufficient credentials (gcloud auth application-default login) and restart the notebook kernel.","Ensure the Dataproc API (dataproc.googleapis.com) is enabled in the target project.","Check organization policies / VPC-SC perimeter that may deny Dataproc reads."],"exampleFix":"// before (shell)\ngcloud auth application-default login  # account without dataproc access\n// after\ngcloud auth application-default login  # account with roles/dataproc.viewer\ngcloud projects add-iam-policy-binding PROJECT_ID \\\n  --member='serviceAccount:SA@PROJECT.iam.gserviceaccount.com' \\\n  --role='roles/dataproc.viewer'","handlingStrategy":"validation","validationCode":"perm = subprocess.run(['gcloud', 'projects', 'get-iam-policy', project, '--flatten=bindings', '--filter=bindings.members:' + member, '--format=value(bindings.role)'], capture_output=True, text=True).stdout\nassert 'dataproc' in perm, 'Missing Dataproc IAM role'","typeGuard":null,"tryCatchPattern":"try:\n    clusters_manager.get_cluster_details()\nexcept ValueError as e:\n    if 'cannot view clusters' in str(e):\n        raise PermissionError('Grant roles/dataproc.viewer on the project') from e\n    raise","preventionTips":["Grant roles/dataproc.viewer to the account/service account before running","Enable the Dataproc API in the project","Verify project_id in ClusterMetadata matches the intended project","Use gcloud auth application-default login with a sufficiently privileged account"],"tags":["gcp","dataproc","iam","permissions","python"],"backgroundTag":"permission-denied","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}