{"record":{"id":"768874afb9d90d6e","repo":"apache/beam","slug":"user-provided-temp-dataset-id-cannot-start-with-r","errorCode":null,"errorMessage":"User provided temp dataset ID cannot start with %r","messagePattern":"User provided temp dataset ID cannot start with %r","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery_tools.py","lineNumber":390,"sourceCode":"            user_agent=\"apache-beam-%s\" % apache_beam.__version__))\n\n    self._unique_row_id = 0\n    # For testing scenarios where we pass in a client we do not want a\n    # randomized prefix for row IDs.\n    self._row_id_prefix = '' if client else uuid.uuid4()\n    self._latency_histogram_metric = Metrics.histogram(\n        self.__class__,\n        'latency_histogram_ms',\n        LinearBucket(0, 20, 3000),\n        BigQueryWrapper.HISTOGRAM_METRIC_LOGGER)\n\n    if temp_dataset_id is not None and temp_table_ref is not None:\n      raise ValueError(\n          'Both a BigQuery temp_dataset_id and a temp_table_ref were specified.'\n          ' Please specify only one of these.')\n\n    if temp_dataset_id and temp_dataset_id.startswith(self.TEMP_DATASET):\n      raise ValueError(\n          'User provided temp dataset ID cannot start with %r' %\n          self.TEMP_DATASET)\n\n    if temp_table_ref is not None:\n      self.temp_table_ref = temp_table_ref\n      self.temp_dataset_id = temp_table_ref.datasetId\n    else:\n      self.temp_table_ref = None\n      self._temporary_table_suffix = uuid.uuid4().hex\n      self.temp_dataset_id = temp_dataset_id or self._get_temp_dataset()\n\n    self.created_temp_dataset = False\n\n  @property\n  def unique_row_id(self):\n    \"\"\"Returns a unique row ID (str) used to avoid multiple insertions.\n\n    If the row ID is provided, BigQuery will make a best effort to not insert","sourceCodeStart":372,"sourceCodeEnd":408,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L372-L408","documentation":"Raised in BigQueryWrapper.__init__ when a user-supplied temp_dataset_id starts with the reserved prefix Beam uses for its own auto-created temp datasets (self.TEMP_DATASET, e.g. 'beam_temp_dataset_'). Beam reserves that namespace to avoid collisions with user datasets.","triggerScenarios":"Calling BigQueryWrapper(temp_dataset_id='beam_temp_dataset_xyz') — any dataset ID whose string starts with the reserved TEMP_DATASET prefix.","commonSituations":"Naming a custom temp dataset to 'blend in' with Beam's convention; copying a previously Beam-generated temp dataset ID from logs into config; template-generated options that reuse an old temp dataset name.","solutions":["Rename the dataset so it does not start with the reserved prefix (e.g. 'my_tmp_dataset').","Strip or replace the 'beam_temp_dataset_' prefix before constructing the wrapper.","Pass neither temp_dataset_id nor temp_table_ref and let Beam allocate a unique temp dataset.","If you truly need that dataset, create it outside Beam's temp mechanism and reference it as a regular dataset."],"exampleFix":"// before\nBigQueryWrapper(temp_dataset_id='beam_temp_dataset_20260912')\n\n// after\nBigQueryWrapper(temp_dataset_id='my_tmp_dataset_20260912')","handlingStrategy":"validation","validationCode":"PREFIX = 'beam_temp_dataset_'\nif temp_dataset_id and temp_dataset_id.startswith(PREFIX):\n    raise ValueError(f\"dataset id {temp_dataset_id!r} uses Beam's reserved prefix\")","typeGuard":"def is_safe_temp_dataset_id(ds_id, reserved_prefix):\n    return bool(ds_id) and not ds_id.startswith(reserved_prefix)","tryCatchPattern":"try:\n    wrapper = BigQueryWrapper(temp_dataset_id=ds_id)\nexcept ValueError as e:\n    if 'cannot start with' in str(e):\n        ds_id = 'my_' + ds_id  # or regenerate the name\n        wrapper = BigQueryWrapper(temp_dataset_id=ds_id)\n    else:\n        raise","preventionTips":["Prefix custom temp datasets with your own project/team tag, not 'beam_temp_dataset_'","Never copy Beam-generated temp dataset IDs from logs into permanent config","Generate unique names with a timestamp or UUID suffix under your own prefix"],"tags":["bigquery","naming","reserved-prefix","temp-dataset"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}