{"record":{"id":"c3631b6336e71b90","repo":"apache/beam","slug":"dataset-does-not-exist-in-your-project-you-have-to-create","errorCode":null,"errorMessage":"Dataset {} does not exist in your project. You have to create table first.","messagePattern":"Dataset (.+?) does not exist in your project\\. You have to create table first\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/testing/load_tests/load_test_metrics_utils.py","lineNumber":510,"sourceCode":"\n    try:\n      self._bq_table = self._client.get_table(table_ref)\n    except NotFound:\n      table = bigquery.Table(table_ref, schema=bq_schemas)\n      self._bq_table = self._client.create_table(table)\n\n  def _update_schema(self):\n    table_schema = self._bq_table.schema\n    if self.schema and len(table_schema) != self.schema:\n      self._bq_table.schema = self._prepare_schema()\n      self._bq_table = self._client.update_table(self._bq_table, [\"schema\"])\n\n  def _get_dataset(self, dataset_name):\n    bq_dataset_ref = self._client.dataset(dataset_name)\n    try:\n      bq_dataset = self._client.get_dataset(bq_dataset_ref)\n    except NotFound:\n      raise ValueError(\n          'Dataset {} does not exist in your project. '\n          'You have to create table first.'.format(dataset_name))\n    return bq_dataset\n\n  def save(self, results):\n    # update schema if needed\n    self._update_schema()\n    return self._client.insert_rows(self._bq_table, results)\n\n\nclass InfluxDBMetricsPublisherOptions(object):\n  def __init__(\n      self,\n      measurement: str,\n      db_name: str,\n      hostname: str,\n      user: Optional[str] = None,\n      password: Optional[str] = None):","sourceCodeStart":492,"sourceCodeEnd":528,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/testing/load_tests/load_test_metrics_utils.py#L492-L528","documentation":"_get_dataset looks up the configured BigQuery dataset and re-raises google.cloud.exceptions.NotFound as a ValueError when the dataset does not exist in the project. The client intentionally does not auto-create datasets; the message tells you to create the dataset (and table) beforehand.","triggerScenarios":"Constructing BigQueryClient pointed at a dataset name that does not exist in the GCP project; typo'd dataset name; wrong project configured on the BigQuery client credentials.","commonSituations":"Running load tests before provisioning the metrics dataset; dataset exists in another project than the one the credentials default to; dataset renamed or deleted.","solutions":["Create the dataset in the target project, e.g. `bq mk <dataset>`","Verify the dataset name spelling in your configuration","Check which project the BigQuery client authenticates to and pass the correct project if needed"],"exampleFix":"// before\nclient = BigQueryClient(schemas, dataset='load_test_metrics')  # dataset missing\n// after: run `bq mk load_test_metrics` first, then construct the client","handlingStrategy":"try-catch","validationCode":"from google.cloud import bigquery\nclient = bigquery.Client()\ndataset_ok = dataset_name in [d.dataset_id for d in client.list_datasets()]","typeGuard":null,"tryCatchPattern":"try:\n    publisher = BigQueryClient(schemas, dataset_name, namespace)\nexcept ValueError as e:\n    if 'does not exist' in str(e):\n        subprocess.run(['bq', 'mk', dataset_name], check=True)\n        publisher = BigQueryClient(schemas, dataset_name, namespace)","preventionTips":["Provision the dataset before launching load tests","Verify the project your credentials default to (gcloud config get-value project)","Pin dataset names in test configuration"],"tags":["bigquery","python","gcp","resource-not-found"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}