{"record":{"id":"b176873bf12b75e1","repo":"apache/beam","slug":"bigquery-table-tablereference-not-found-if-you-wanted-to","errorCode":null,"errorMessage":"BigQuery table \" + tableReference + \" not found. If you wanted to automatically create the table, set the create disposition to CREATE_IF_NEEDED and specify a schema.","messagePattern":"BigQuery table \" \\+ tableReference \\+ \" not found\\. If you wanted to automatically create the table, set the create disposition to CREATE_IF_NEEDED and specify a schema\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiDynamicDestinationsTableRow.java","lineNumber":186,"sourceCode":"    final com.google.cloud.bigquery.storage.v1.TableSchema protoTableSchema;\n    final Supplier<byte[]> getSchemaHash;\n    final TableRowToStorageApiProto.SchemaInformation schemaInformation;\n    final Descriptor descriptor;\n    final @Nullable Descriptor cdcDescriptor;\n\n    TableRowConverter(\n        DestinationT destination,\n        DatasetService datasetService,\n        @Nullable TableSchema localTableSchema)\n        throws Exception {\n      this.tableReference = getTable(destination).getTableReference();\n      if (localTableSchema == null) {\n        // If the table already exists, then try and fetch the schema from the existing\n        // table.\n        localTableSchema = SCHEMA_CACHE.getSchema(tableReference, datasetService);\n        if (localTableSchema == null) {\n          if (createDisposition == CreateDisposition.CREATE_NEVER) {\n            throw new RuntimeException(\n                \"BigQuery table \"\n                    + tableReference\n                    + \" not found. If you wanted to \"\n                    + \"automatically create the table, set the create disposition to CREATE_IF_NEEDED and specify a \"\n                    + \"schema.\");\n          } else {\n            throw new RuntimeException(\n                \"Schema must be set for table \"\n                    + tableReference\n                    + \" when writing TableRows using Storage API and \"\n                    + \"using a create disposition of CREATE_IF_NEEDED.\");\n          }\n        }\n      } else {\n        // Make sure we register this schema with the cache, unless there's already a more\n        // up-to-date schema.\n        localTableSchema =\n            MoreObjects.firstNonNull(","sourceCodeStart":168,"sourceCodeEnd":204,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiDynamicDestinationsTableRow.java#L168-L204","documentation":"RuntimeException thrown by StorageApiDynamicDestinationsTableRow.TableRowConverter when the destination BigQuery table's schema cannot be fetched and the write's create disposition is CREATE_NEVER. Since the library is forbidden from creating the table, a missing table is fatal.","triggerScenarios":"Writing TableRows via BigQueryIO with Storage API and CreateDisposition.CREATE_NEVER while the target table does not exist (or is not visible to the caller), so SCHEMA_CACHE returns null and no schema can be resolved.","commonSituations":"Typo in dataset/table name; table created in a different project than configured; running with credentials/permissions that cannot see the table; assuming CREATE_NEVER auto-creates tables (it never does); renamed or dropped tables between pipeline setup and execution.","solutions":["Create the table in BigQuery (or fix the table reference spelling/project) before running the pipeline","Change the write to use CreateDisposition.CREATE_IF_NEEDED and supply a schema via withSchema(...)","Verify the credentials can access the table in the specified project","Use bq show or the console to confirm the fully-qualified tableReference exists"],"exampleFix":"// before\nBigQueryIO.writeTableRows().to(tableSpec).withCreateDisposition(CreateDisposition.CREATE_NEVER)\n// after\nBigQueryIO.writeTableRows().to(tableSpec)\n  .withCreateDisposition(CreateDisposition.CREATE_IF_NEEDED)\n  .withSchema(tableSchema);","handlingStrategy":"validation","validationCode":"TableId id = TableId.of(project, dataset, table);\nif (bigquery.getTable(id) == null) {\n  throw new IllegalStateException(\"Table \" + id + \" does not exist; create it or use CREATE_IF_NEEDED\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"BigQuery table\") && e.getMessage().contains(\"not found\")) {\n    bigquery.create(TableInfo.newBuilder(tableId, TableDefinition.of(StandardTableDefinition.of(schema))).build());\n    // resubmit the pipeline\n  } else { throw e; }\n}","preventionTips":["Always verify the fully-qualified table (project:dataset.table) exists before a CREATE_NEVER write","Double-check dataset/table spelling and target project","Use CREATE_IF_NEEDED with an explicit schema when auto-creation is acceptable","Confirm credentials can see the table (IAM + project scope)"],"tags":["bigquery","resource-not-found","storage-write","gcp"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}