{"record":{"id":"e6fa14dc2184b0b7","repo":"apache/beam","slug":"unknown-cluster-identifier-s-cannot-create-or-reusea","errorCode":null,"errorMessage":"Unknown cluster identifier: %s. Cannot create or reusea Dataproc cluster.","messagePattern":"Unknown cluster identifier: (.+?)\\. Cannot create or reusea Dataproc cluster\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/interactive/interactive_beam.py","lineNumber":433,"sourceCode":"  # to run.\n  # DATAPROC_IMAGE_VERSION = '2.0.XX-debian10'\n\n  def __init__(self) -> None:\n    self.dataproc_cluster_managers: dict[ClusterMetadata,\n                                         DataprocClusterManager] = {}\n    self.master_urls: dict[str, ClusterMetadata] = {}\n    self.pipelines: dict[beam.Pipeline, DataprocClusterManager] = {}\n    self.default_cluster_metadata: Optional[ClusterMetadata] = None\n\n  def create(\n      self, cluster_identifier: ClusterIdentifier) -> DataprocClusterManager:\n    \"\"\"Creates a Dataproc cluster manager provisioned for the cluster\n    identified. If the cluster is known, returns an existing cluster manager.\n    \"\"\"\n    # Try to get some not-None cluster metadata.\n    cluster_metadata = self.cluster_metadata(cluster_identifier)\n    if not cluster_metadata:\n      raise ValueError(\n          'Unknown cluster identifier: %s. Cannot create or reuse'\n          'a Dataproc cluster.')\n    if not cluster_metadata.region:\n      _LOGGER.info(\n          'No region information was detected, defaulting Dataproc cluster '\n          'region to: us-central1.')\n      cluster_metadata.region = 'us-central1'\n    elif cluster_metadata.region == 'global':\n      # The global region is unsupported as it will be eventually deprecated.\n      raise ValueError('Clusters in the global region are not supported.')\n    # else use the provided region.\n    if (cluster_metadata.num_workers and\n        cluster_metadata.num_workers < self.DATAPROC_MINIMUM_WORKER_NUM):\n      _LOGGER.info(\n          'At least %s workers are required for a cluster, defaulting to %s.',\n          self.DATAPROC_MINIMUM_WORKER_NUM,\n          self.DATAPROC_MINIMUM_WORKER_NUM)\n      cluster_metadata.num_workers = self.DATAPROC_MINIMUM_WORKER_NUM","sourceCodeStart":415,"sourceCodeEnd":451,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/interactive/interactive_beam.py#L415-L451","documentation":"DataprocClusterManager.create() first resolves the given cluster_identifier into ClusterMetadata; if none is found it raises this ValueError and refuses to create or reuse a Dataproc cluster. Note the message template is missing its %s argument, so it prints literally with '%s' — a known bug in Beam.","triggerScenarios":"Calling ClustersManager.create(cluster_identifier=...) where cluster_identifier is not a known ClouderMetadata (not registered via record/previous detection) and cannot be derived from options, so self.cluster_metadata(cluster_identifier) returns None.","commonSituations":"Passing a cluster name string without matching project/region/label metadata registered; reusing a manager across notebook restarts where the metadata cache was lost; passing an integer/None identifier; typo in the cluster identifier.","solutions":["Register the cluster metadata first (e.g. Cache.record_cluster_metadata / ic.record_cluster_metadata_if_allowed) before calling create().","Pass a valid identifier type: a ClusterMetadata, or configure the cluster name/project/region via interactive options.","Print existing known metadata (Cache.list_clusters_metadata) to find a valid identifier.","Inspect self.cluster_metadata(cluster_identifier) to see why resolution returns None (wrong region/project).","If the literal '%s' appears in the message, note the missing identifier is your argument — enable debug logging to confirm."],"exampleFix":"// before\nmanager = ClustersManager.create(cluster_identifier='some-cluster')\n// after\nfrom apache_beam.runners.interactive.dataproc.types import ClusterMetadata\nCache.record_cluster_metadata('session-1', ClusterMetadata(cluster_name='some-cluster', project_id='my-proj', region='us-central1'))\nmanager = ClustersManager.create(cluster_identifier=ClusterMetadata(cluster_name='some-cluster', project_id='my-proj', region='us-central1'))","handlingStrategy":"validation","validationCode":"from apache_beam.runners.interactive.dataproc.cluster_manager import Cache\nmetadata = Cache.get_cluster_metadata(identifier)\nassert metadata is not None, f'Register cluster metadata for {identifier!r} before create()'","typeGuard":"def identifier_is_known(identifier) -> bool:\n    from apache_beam.runners.interactive.dataproc.cluster_manager import ClustersManager\n    return ClustersManager.cluster_metadata(identifier) is not None","tryCatchPattern":"try:\n    manager = ClustersManager.create(cluster_identifier=identifier)\nexcept ValueError as e:\n    if 'Unknown cluster identifier' in str(e):\n        manager = provision_with_explicit_metadata(identifier)\n    else:\n        raise","preventionTips":["Record ClusterMetadata (name/project/region) before calling create()","Pass a ClusterMetadata object instead of a bare string when unsure","Inspect Cache.list_clusters_metadata for known identifiers","Remember the error message prints a literal %s due to a Beam bug; check debug logs for the actual identifier"],"tags":["gcp","dataproc","python","invalid-identifier"],"backgroundTag":"entity-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}