{"record":{"id":"969e2c618b58b8d8","repo":"apache/beam","slug":"table-s-s-s-requires-a-schema-none-can-be-inferred-because","errorCode":null,"errorMessage":"Table %s:%s.%s requires a schema. None can be inferred because the table does not exist.","messagePattern":"Table (.+?):(.+?)\\.(.+?) requires a schema\\. None can be inferred because the table does not exist\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/bigquery_tools.py","lineNumber":1202,"sourceCode":"\n    # If table exists already then handle the semantics for WRITE_EMPTY and\n    # WRITE_TRUNCATE write dispositions.\n    if found_table and write_disposition in (\n        BigQueryDisposition.WRITE_EMPTY, BigQueryDisposition.WRITE_TRUNCATE):\n      # Delete the table and recreate it (later) if WRITE_TRUNCATE was\n      # specified.\n      if write_disposition == BigQueryDisposition.WRITE_TRUNCATE:\n        self._delete_table(project_id, dataset_id, table_id)\n      elif (write_disposition == BigQueryDisposition.WRITE_EMPTY and\n            not self._is_table_empty(project_id, dataset_id, table_id)):\n        raise RuntimeError(\n            'Table %s:%s.%s is not empty but write disposition is WRITE_EMPTY.'\n            % (project_id, dataset_id, table_id))\n\n    # Create a new table potentially reusing the schema from a previously\n    # found table in case the schema was not specified.\n    if schema is None and found_table is None:\n      raise RuntimeError(\n          'Table %s:%s.%s requires a schema. None can be inferred because the '\n          'table does not exist.' % (project_id, dataset_id, table_id))\n    if found_table and write_disposition != BigQueryDisposition.WRITE_TRUNCATE:\n      return found_table\n    else:\n      created_table = None\n      try:\n        created_table = self._create_table(\n            project_id=project_id,\n            dataset_id=dataset_id,\n            table_id=table_id,\n            schema=schema or found_table.schema,\n            additional_parameters=additional_create_parameters)\n      except HttpError as exn:\n        if exn.status_code == 409:\n          _LOGGER.debug(\n              'Skipping Creation. Table %s:%s.%s already exists.' %\n              (project_id, dataset_id, table_id))","sourceCodeStart":1184,"sourceCodeEnd":1220,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L1184-L1220","documentation":"Raised by `get_or_create_table` when no schema was supplied and the table does not exist, so Beam has no schema to create the table with and cannot infer one. Creating a BigQuery table requires a schema; with CREATE_IF_NEEDED and a missing table, a `schema=None` argument makes the operation impossible.","triggerScenarios":"Calling `get_or_create_table(project, dataset, table_id, schema=None, ...)` where the table is absent, expecting the schema to be discovered from the existing table; passing a schema argument that resolves to None (e.g. an unset pipeline option or an empty variable).","commonSituations":"Copy code written for existing tables applied to new tables; schema loaded conditionally and silently None when file/config missing; dynamic destinations where schema lookup returns None for unseen destination.","solutions":["Pass an explicit schema (apache_beam.io.gcp.internal.clients.bigquery TableSchema or dict) when writing to a table that may not exist.","Load the schema from a JSON file via bigquery_tools.parse_table_schema_from_json and verify it isn't None before calling.","Create the table in advance (bq mk --schema=...) so the existing table's schema can be reused.","Add a guard: if schema is None, fail fast in your own code with a clear message about which destination lacks a schema."],"exampleFix":"// before\nschema = get_schema_option()  # may be None\nget_or_create_table(p, d, t, schema, CREATE_IF_NEEDED, WRITE_APPEND)\n// after\nschema = get_schema_option() or parse_table_schema_from_json('schema.json')\nassert schema is not None, 'schema required for non-existent table'\nget_or_create_table(p, d, t, schema, CREATE_IF_NEEDED, WRITE_APPEND)","handlingStrategy":"validation","validationCode":"def require_schema(schema, dest):\n    if schema is None:\n        raise ValueError(f'A schema is required for destination {dest}; table may not exist yet.')\n    return schema\n# call before get_or_create_table with CREATE_IF_NEEDED","typeGuard":"def has_schema(schema):\n    return schema is not None and getattr(schema, 'fields', None)","tryCatchPattern":"try:\n    get_or_create_table(p, d, t, schema, CREATE_IF_NEEDED, WRITE_APPEND)\nexcept RuntimeError as e:\n    if 'requires a schema' in str(e):\n        schema = load_schema_from_config(dest)\n        get_or_create_table(p, d, t, schema, CREATE_IF_NEEDED, WRITE_APPEND)\n    else:\n        raise","preventionTips":["Never rely on schema inference for tables that may not exist.","Ship schema JSON with the pipeline and load it deterministically.","Fail fast in your own config-loading code when a schema resolves to None."],"tags":["bigquery","python","gcp","schema"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}