{"record":{"id":"dace97842dfdfa65","repo":"apache/beam","slug":"schema-must-be-set-for-table-tablereference-when-writing","errorCode":null,"errorMessage":"Schema must be set for table \" + tableReference + \" when writing TableRows using Storage API and using a create disposition of CREATE_IF_NEEDED.","messagePattern":"Schema must be set for table \" \\+ tableReference \\+ \" when writing TableRows using Storage API and using a create disposition of CREATE_IF_NEEDED\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiDynamicDestinationsTableRow.java","lineNumber":193,"sourceCode":"        DestinationT destination,\n        DatasetService datasetService,\n        @Nullable TableSchema localTableSchema)\n        throws Exception {\n      this.tableReference = getTable(destination).getTableReference();\n      if (localTableSchema == null) {\n        // If the table already exists, then try and fetch the schema from the existing\n        // table.\n        localTableSchema = SCHEMA_CACHE.getSchema(tableReference, datasetService);\n        if (localTableSchema == null) {\n          if (createDisposition == CreateDisposition.CREATE_NEVER) {\n            throw new RuntimeException(\n                \"BigQuery table \"\n                    + tableReference\n                    + \" not found. If you wanted to \"\n                    + \"automatically create the table, set the create disposition to CREATE_IF_NEEDED and specify a \"\n                    + \"schema.\");\n          } else {\n            throw new RuntimeException(\n                \"Schema must be set for table \"\n                    + tableReference\n                    + \" when writing TableRows using Storage API and \"\n                    + \"using a create disposition of CREATE_IF_NEEDED.\");\n          }\n        }\n      } else {\n        // Make sure we register this schema with the cache, unless there's already a more\n        // up-to-date schema.\n        localTableSchema =\n            MoreObjects.firstNonNull(\n                SCHEMA_CACHE.putSchemaIfAbsent(tableReference, localTableSchema), localTableSchema);\n      }\n      this.tableSchema = localTableSchema;\n      this.protoTableSchema = TableRowToStorageApiProto.schemaToProtoTableSchema(tableSchema);\n      this.getSchemaHash =\n          Suppliers.memoize(() -> TableRowToStorageApiProto.tableSchemaHash(this.protoTableSchema));\n      schemaInformation =","sourceCodeStart":175,"sourceCodeEnd":211,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiDynamicDestinationsTableRow.java#L175-L211","documentation":"RuntimeException thrown by StorageApiDynamicDestinationsTableRow.TableRowConverter when writing TableRows with the Storage API and CreateDisposition.CREATE_IF_NEEDED, but no schema was provided and the table does not exist. TableRows are schema-less, so BigQueryIO requires an explicit schema to create the table.","triggerScenarios":"BigQueryIO.writeTableRows().withCreateDisposition(CREATE_IF_NEEDED) with no .withSchema(...) call and the destination table absent, so dynamicDestinations.getSchema returns null.","commonSituations":"Switching from Avro/TableRow-based writes (which carry schema) to raw TableRows and forgetting withSchema; relying on a table that was later deleted; dynamic destinations where getSchema() returns null for a new destination.","solutions":["Add .withSchema(Schema or TableSchema) to the BigQueryIO write transform","Pre-create the destination table with the desired schema so the converter can fetch it","For DynamicDestinations, ensure getSchema(destination) returns a non-null schema for every destination"],"exampleFix":"// before\nBigQueryIO.writeTableRows().to(tableSpec).withCreateDisposition(CreateDisposition.CREATE_IF_NEEDED)\n// after\nBigQueryIO.writeTableRows().to(tableSpec)\n  .withCreateDisposition(CreateDisposition.CREATE_IF_NEEDED)\n  .withSchema(new TableSchema().setFields(fields));","handlingStrategy":"validation","validationCode":"if (createDisposition == CreateDisposition.CREATE_IF_NEEDED && writeSchema == null && bigquery.getTable(tableId) == null) {\n  throw new IllegalArgumentException(\"Provide withSchema(...) for STORAGE_API TableRow writes\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  result = writeResult;\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Schema must be set for table\")) {\n    // rebuild the transform with .withSchema(...) and resubmit\n  } else { throw e; }\n}","preventionTips":["Always call withSchema(...) when writing TableRows with STORAGE_API methods","For DynamicDestinations, ensure getSchema() is non-null for every destination","Pre-create destination tables so the schema can be fetched from BigQuery"],"tags":["bigquery","schema","storage-write","gcp"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}