{"record":{"id":"435acb7ca25f0169","repo":"apache/beam","slug":"table-s-s-s-is-not-empty-but-write-disposition-is-write","errorCode":null,"errorMessage":"Table %s:%s.%s is not empty but write disposition is WRITE_EMPTY.","messagePattern":"Table (.+?):(.+?)\\.(.+?) is not empty but write disposition is WRITE_EMPTY\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery_tools.py","lineNumber":1195,"sourceCode":"      if exn.status_code == 404:\n        if create_disposition == BigQueryDisposition.CREATE_NEVER:\n          raise RuntimeError(\n              'Table %s:%s.%s not found but create disposition is CREATE_NEVER.'\n              % (project_id, dataset_id, table_id))\n      else:\n        raise\n\n    # If table exists already then handle the semantics for WRITE_EMPTY and\n    # WRITE_TRUNCATE write dispositions.\n    if found_table and write_disposition in (\n        BigQueryDisposition.WRITE_EMPTY, BigQueryDisposition.WRITE_TRUNCATE):\n      # Delete the table and recreate it (later) if WRITE_TRUNCATE was\n      # specified.\n      if write_disposition == BigQueryDisposition.WRITE_TRUNCATE:\n        self._delete_table(project_id, dataset_id, table_id)\n      elif (write_disposition == BigQueryDisposition.WRITE_EMPTY and\n            not self._is_table_empty(project_id, dataset_id, table_id)):\n        raise RuntimeError(\n            'Table %s:%s.%s is not empty but write disposition is WRITE_EMPTY.'\n            % (project_id, dataset_id, table_id))\n\n    # Create a new table potentially reusing the schema from a previously\n    # found table in case the schema was not specified.\n    if schema is None and found_table is None:\n      raise RuntimeError(\n          'Table %s:%s.%s requires a schema. None can be inferred because the '\n          'table does not exist.' % (project_id, dataset_id, table_id))\n    if found_table and write_disposition != BigQueryDisposition.WRITE_TRUNCATE:\n      return found_table\n    else:\n      created_table = None\n      try:\n        created_table = self._create_table(\n            project_id=project_id,\n            dataset_id=dataset_id,\n            table_id=table_id,","sourceCodeStart":1177,"sourceCodeEnd":1213,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L1177-L1213","documentation":"Raised by `get_or_create_table` when the table exists, `write_disposition` is `WRITE_EMPTY`, and `_is_table_empty` finds rows in the table. WRITE_EMPTY semantics require the destination to be empty for the write to proceed, so Beam fails loudly rather than silently appending. This is expected BigQuery write-disposition behavior surfaced as a RuntimeError.","triggerScenarios":"Calling `get_or_create_table` (or a WriteToBigQuery sink configured with WRITE_EMPTY) against a table that already contains data — e.g. rerunning a batch job that wrote to the same table previously.","commonSituations":"Rerunning a backfill without clearing the destination; writing periodic batches to a table intended for one-shot load; misunderstanding WRITE_EMPTY (fails if non-empty) vs WRITE_TRUNCATE (replaces) vs WRITE_APPEND.","solutions":["Switch `write_disposition` to `WRITE_APPEND` if you want to add data to a non-empty table.","Use `WRITE_TRUNCATE` if the pipeline output should fully replace the table contents.","If the table should be empty, clear it first (bq truncate, or delete via console) and rerun.","Audit pipeline idempotency: a rerun hitting this means the job already wrote once — check for duplicate execution before retrying."],"exampleFix":"// before\nget_or_create_table(p, d, t, schema, CREATE_IF_NEEDED, BigQueryDisposition.WRITE_EMPTY)\n// after\nget_or_create_table(p, d, t, schema, CREATE_IF_NEEDED, BigQueryDisposition.WRITE_APPEND)","handlingStrategy":"validation","validationCode":"from google.cloud import bigquery\nclient = bigquery.Client()\ndef table_row_count(project, dataset, table):\n    t = client.get_table(f'{project}.{dataset}.{table}')\n    return t.num_rows\n# choose disposition based on count\ndisposition = WRITE_EMPTY if table_row_count(p, d, t) == 0 else WRITE_APPEND","typeGuard":null,"tryCatchPattern":"try:\n    get_or_create_table(p, d, t, schema, CREATE_IF_NEEDED, WRITE_EMPTY)\nexcept RuntimeError as e:\n    if 'WRITE_EMPTY' in str(e):\n        raise DecisionNeeded('table non-empty; choose WRITE_APPEND or WRITE_TRUNCATE') from e\n    raise","preventionTips":["Decide WRITE_EMPTY vs WRITE_APPEND explicitly based on table state, not by default.","Make batch jobs idempotent (WRITE_TRUNCATE or per-run destination tables).","Check num_rows before one-shot loads into shared tables."],"tags":["bigquery","python","gcp","disposition"],"backgroundTag":"invalid-state-transition","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}