{"record":{"id":"35e7110bf95ede6b","repo":"apache/beam","slug":"table-s-s-s-not-found-but-create-disposition-is-create-never","errorCode":null,"errorMessage":"Table %s:%s.%s not found but create disposition is CREATE_NEVER.","messagePattern":"Table (.+?):(.+?)\\.(.+?) not found but create disposition is CREATE_NEVER\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery_tools.py","lineNumber":1179,"sourceCode":"\n    Returns:\n      A bigquery.Table instance if table was found or created.\n\n    Raises:\n      `RuntimeError`: For various mismatches between the state of the table and\n        the create/write dispositions passed in. For example if the table is not\n        empty and WRITE_EMPTY was specified then an error will be raised since\n        the table was expected to be empty.\n    \"\"\"\n    from apache_beam.io.gcp.bigquery import BigQueryDisposition\n\n    found_table = None\n    try:\n      found_table = self.get_table(project_id, dataset_id, table_id)\n    except HttpError as exn:\n      if exn.status_code == 404:\n        if create_disposition == BigQueryDisposition.CREATE_NEVER:\n          raise RuntimeError(\n              'Table %s:%s.%s not found but create disposition is CREATE_NEVER.'\n              % (project_id, dataset_id, table_id))\n      else:\n        raise\n\n    # If table exists already then handle the semantics for WRITE_EMPTY and\n    # WRITE_TRUNCATE write dispositions.\n    if found_table and write_disposition in (\n        BigQueryDisposition.WRITE_EMPTY, BigQueryDisposition.WRITE_TRUNCATE):\n      # Delete the table and recreate it (later) if WRITE_TRUNCATE was\n      # specified.\n      if write_disposition == BigQueryDisposition.WRITE_TRUNCATE:\n        self._delete_table(project_id, dataset_id, table_id)\n      elif (write_disposition == BigQueryDisposition.WRITE_EMPTY and\n            not self._is_table_empty(project_id, dataset_id, table_id)):\n        raise RuntimeError(\n            'Table %s:%s.%s is not empty but write disposition is WRITE_EMPTY.'\n            % (project_id, dataset_id, table_id))","sourceCodeStart":1161,"sourceCodeEnd":1197,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L1161-L1197","documentation":"Raised by `get_or_create_table` when `get_table` returns HTTP 404 (table missing) while `create_disposition` is `CREATE_NEVER`. Because CREATE_NEVER forbids creating the table, Beam propagates a RuntimeError instead of silently creating one. The write can never proceed under this configuration.","triggerScenarios":"Writing to BigQuery with `create_disposition=BigQueryDisposition.CREATE_NEVER` while the target table `project:dataset.table` does not exist (typo, wrong dataset, table created in a different project).","commonSituations":"Typo'd table or dataset names in pipeline options; pointing a staging pipeline at a fresh project without migrations; case-sensitivity confusion between projects; environment misconfig (dev vs prod).","solutions":["Create the table before the pipeline runs, or change `create_disposition` to `CREATE_IF_NEEDED` and supply a schema.","Verify the project_id, dataset_id, and table_id spelling against the actual BigQuery resource (bq show <project>:<dataset>.<table>).","Check pipeline options/environment — you may be pointed at the wrong project (dev vs prod).","If the table is created by an upstream job, add a dependency/precedence guarantee so it exists before this write starts."],"exampleFix":"// before\ntable_info = bq_tools.get_or_create_table(project, dataset, table, schema, BigQueryDisposition.CREATE_NEVER, BigQueryDisposition.WRITE_APPEND)\n// after\ntable_info = bq_tools.get_or_create_table(project, dataset, table, schema, BigQueryDisposition.CREATE_IF_NEEDED, BigQueryDisposition.WRITE_APPEND)","handlingStrategy":"validation","validationCode":"from google.api_core.exceptions import NotFound\nfrom google.cloud import bigquery\nclient = bigquery.Client()\ndef table_exists(project, dataset, table):\n    try:\n        client.get_table(f'{project}.{dataset}.{table}')\n        return True\n    except NotFound:\n        return False\n# require table_exists(...) true before using CREATE_NEVER","typeGuard":null,"tryCatchPattern":"try:\n    get_or_create_table(p, d, t, schema, CREATE_NEVER, WRITE_APPEND)\nexcept RuntimeError as e:\n    if 'CREATE_NEVER' in str(e):\n        log.error('destination table %s:%s.%s missing', p, d, t)\n    raise","preventionTips":["Use CREATE_NEVER only when an upstream process guarantees table creation.","Verify table identifiers against `bq show` before launching batch jobs.","Check --project / environment configuration to avoid dev/prod mix-ups."],"tags":["bigquery","python","gcp","disposition"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}