{"record":{"id":"d92a04e1746027a6","repo":"apache/beam","slug":"table-not-found-bigquerytablesource","errorCode":null,"errorMessage":"Table not found: ","messagePattern":"Table not found: ","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableSource.java","lineNumber":81,"sourceCode":"\n  @Override\n  protected TableReference getTableToExtract(BigQueryOptions bqOptions) throws IOException {\n    return tableDef.getTableReference(bqOptions);\n  }\n\n  @Override\n  public synchronized long getEstimatedSizeBytes(PipelineOptions options) throws Exception {\n    Long maybeNumBytes = tableSizeBytes.get();\n    if (maybeNumBytes != null) {\n      return maybeNumBytes;\n    } else {\n      BigQueryOptions bqOptions = options.as(BigQueryOptions.class);\n      TableReference tableRef = tableDef.getTableReference(bqOptions);\n      try (DatasetService datasetService = bqServices.getDatasetService(bqOptions)) {\n        Table table = datasetService.getTable(tableRef);\n\n        if (table == null) {\n          throw new IllegalStateException(\"Table not found: \" + table);\n        }\n\n        Long numBytes = table.getNumBytes();\n        if (numBytes == null) {\n          // Tables that don't report storage statistics, e.g. Lakehouse runtime catalog\n          // (BigLake metastore) tables.\n          numBytes = 0L;\n        }\n        if (table.getStreamingBuffer() != null\n            && table.getStreamingBuffer().getEstimatedBytes() != null) {\n          numBytes += table.getStreamingBuffer().getEstimatedBytes().longValue();\n        }\n\n        tableSizeBytes.compareAndSet(null, numBytes);\n        return numBytes;\n      }\n    }\n  }","sourceCodeStart":63,"sourceCodeEnd":99,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableSource.java#L63-L99","documentation":"BigQueryTableSource.getEstimatedSizeBytes estimates source size by reading the table's numBytes metadata from DatasetService.getTable. If the table lookup returns null, an IllegalStateException is thrown. Like the sibling error in BigQueryStorageTableSource, the message concatenates the null table, so it renders as 'Table not found: null'.","triggerScenarios":"Cost/size estimation phase of a BOUNDED_READ (extract/QUERY) BigQueryIO read when the referenced table is missing, in the wrong project, invisible to the credentials, or deleted after job graph construction.","commonSituations":"Typo'd dataset/table; pipeline built against a dev dataset but run against prod credentials; service account lacking metadataViewer; BigLake/external tables in datasets the caller can't describe.","solutions":["Confirm the table exists with the pipeline's credentials (`bq show project:dataset.table`)","Set --project/--bigQueryProject to the correct project containing the table","Grant the service account bigquery.tables.get / roles/bigquery.metadataViewer on the dataset","Guard table lifecycle: ensure the table isn't dropped between pipeline construction and execution (recreate or skip reads of ephemeral tables)"],"exampleFix":"// before\npipeline.apply(BigQueryIO.readTableRows().from(\"proj:ds.\" + tableName)); // tableName may be stale\n// after\nTableReference ref = TableReference.from(\"proj\", \"ds\", tableName);\n// pre-flight check with same credentials:\n// bqClient.getTable(ref) != null  ->  else fail fast with a clear message\npipeline.apply(BigQueryIO.readTableRows().from(ref));","handlingStrategy":"validation","validationCode":"BigQuery bq = BigQueryOptions.getDefaultInstance().getService();\nif (bq.getTable(\"proj\", \"ds\", \"tbl\") == null) {\n  throw new IllegalArgumentException(\"proj:ds.tbl not found or not accessible\");\n}","typeGuard":"boolean tableExists(BigQuery bq, TableReference ref) {\n  return ref != null && bq.getTable(ref) != null;\n}","tryCatchPattern":"try {\n  source.getEstimatedSizeBytes(options);\n} catch (IllegalStateException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Table not found\")) {\n    // fall back to a conservative size estimate or fail fast with a clear message\n  }\n  throw e;\n}","preventionTips":["Verify table existence before composing the pipeline (fail fast with a clear message)","Keep table references fully qualified and sourced from validated config","Ensure the pipeline's service account has table metadata permissions","Re-check existence for long-lived pipelines referencing ephemeral tables"],"tags":["bigquery","resource-not-found","size-estimation","gcp"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}