{"record":{"id":"9ac344cc52567dbd","repo":"apache/beam","slug":"you-cannot-create-a-cluster-in-project","errorCode":null,"errorMessage":"You cannot create a cluster in project: {}","messagePattern":"You cannot create a cluster in project: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":403,"severity":"error","filePath":"sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py","lineNumber":149,"sourceCode":"      return\n    try:\n      self._cluster_client.create_cluster(\n          request={\n              'project_id': self.cluster_metadata.project_id,\n              'region': self.cluster_metadata.region,\n              'cluster': cluster\n          })\n    except Exception as e:\n      if e.code == 409:\n        _LOGGER.info(\n            'Cluster %s already exists. Continuing...',\n            self.cluster_metadata.cluster_name)\n      elif e.code == 403:\n        _LOGGER.error(\n            'Due to insufficient project permissions, '\n            'unable to create cluster: %s',\n            self.cluster_metadata.cluster_name)\n        raise ValueError(\n            'You cannot create a cluster in project: {}'.format(\n                self.cluster_metadata.project_id))\n      elif e.code == 501:\n        _LOGGER.error(\n            'Invalid region provided: %s', self.cluster_metadata.region)\n        raise ValueError(\n            'Region {} does not exist!'.format(self.cluster_metadata.region))\n      else:\n        _LOGGER.error(\n            'Unable to create cluster: %s', self.cluster_metadata.cluster_name)\n        raise e\n    else:\n      _LOGGER.info(\n          'Cluster created successfully: %s',\n          self.cluster_metadata.cluster_name)\n      self._staging_directory = self.get_staging_location()\n      master_url, dashboard = self.get_master_url_and_dashboard()\n      self.cluster_metadata.master_url = master_url","sourceCodeStart":131,"sourceCodeEnd":167,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/interactive/dataproc/dataproc_cluster_manager.py#L131-L167","documentation":"When the Dataproc API returns HTTP 403 during create_cluster, the manager translates it into ValueError('You cannot create a cluster in project: ...'), indicating the credentials lack permission to create clusters in that project.","triggerScenarios":"Calling create_cluster (via create_flink_cluster) where the service account/user lacks dataproc.clusters.create on the project; 403 from the Dataproc regions.create API call.","commonSituations":"Using default credentials without Dataproc roles; wrong project id; org policy or service account restrictions; missing iam.serviceAccountUser role for the Dataproc service agent.","solutions":["Grant the caller the Dataproc Admin/Editor role (dataproc.clusters.create) on the project.","Verify project_id in ClusterMetadata is the intended, correct project.","Check that the Dataproc service agent has iam.serviceAccountUser permission on the compute service account.","Confirm credentials (gcloud auth application-default login / service account key) belong to an account with access."],"exampleFix":"// before\n# caller lacks roles; fails with 403\ncluster = manager.create_flink_cluster()\n// after\n# grant: gcloud projects add-iam-policy-binding PROJECT --member=user:me@example.com --role=roles/dataproc.editor\ncluster = manager.create_flink_cluster()","handlingStrategy":"try-catch","validationCode":"# preflight IAM check\nfrom google.cloud import dataproc_v1\nclient = dataproc_v1.ClusterControllerClient(client_options={'api_endpoint': f'{region}-dataproc.googleapis.com:443'})\n# caller must have dataproc.clusters.create; test with a dry list call\ntry:\n    client.list_clusters(request={'project_id': project, 'region': region})\nexcept Exception as e:\n    check_permission(e)","typeGuard":null,"tryCatchPattern":"try:\n    cluster = manager.create_flink_cluster()\nexcept ValueError as e:\n    if 'You cannot create a cluster in project' in str(e):\n        grant_dataproc_roles() or switch project/service account\n    else:\n        raise","preventionTips":["Run gcloud auth application-default login with an account holding roles/dataproc.editor.","Confirm the project id and org policies allow cluster creation.","Ensure the Dataproc service agent has serviceAccountUser on the compute SA."],"tags":["apache-beam","dataproc","gcp","permissions","cluster"],"backgroundTag":"insufficient-permissions","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}