{"record":{"id":"660d45594086fe9d","repo":"apache/beam","slug":"cannot-start-an-export-job-since-table-s-does-not-exist","errorCode":null,"errorMessage":"Cannot start an export job since table %s does not exist","messagePattern":"Cannot start an export job since table (.+?) does not exist","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java","lineNumber":118,"sourceCode":"    }\n\n    public ExtractResult(\n        TableSchema schema,\n        List<ResourceId> extractedFiles,\n        @Nullable List<MatchResult.Metadata> metadata) {\n      this.schema = schema;\n      this.extractedFiles = extractedFiles;\n      this.metadata = metadata;\n    }\n  }\n\n  protected ExtractResult extractFiles(PipelineOptions options) throws Exception {\n    BigQueryOptions bqOptions = options.as(BigQueryOptions.class);\n    TableReference tableToExtract = getTableToExtract(bqOptions);\n    try (BigQueryServices.DatasetService datasetService = bqServices.getDatasetService(bqOptions)) {\n      Table table = datasetService.getTable(tableToExtract);\n      if (table == null) {\n        throw new IOException(\n            String.format(\n                \"Cannot start an export job since table %s does not exist\",\n                BigQueryHelpers.toTableSpec(tableToExtract)));\n      }\n      // emit this table ID as a lineage source\n      Lineage.getSources()\n          .add(\"bigquery\", BigQueryHelpers.dataCatalogSegments(tableToExtract, bqOptions));\n\n      TableSchema schema = table.getSchema();\n      JobService jobService = bqServices.getJobService(bqOptions);\n      String extractJobId =\n          BigQueryResourceNaming.createJobIdPrefix(options.getJobName(), stepUuid, JobType.EXPORT);\n      final String extractDestinationDir =\n          resolveTempLocation(bqOptions.getTempLocation(), \"BigQueryExtractTemp\", stepUuid);\n      String bqLocation =\n          BigQueryHelpers.getDatasetLocation(\n              datasetService, tableToExtract.getProjectId(), tableToExtract.getDatasetId());\n      String bqProjectId =","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java#L100-L136","documentation":"BigQuerySourceBase.extractFiles starts an export (extract) job to dump the source table to GCS, but first fetches the table via DatasetService.getTable; if the table is null (does not exist), it throws this IOException with the table spec. The read cannot proceed without a table to export.","triggerScenarios":"Reading from a BigQuery table (BigQueryIO.read / TableSource) whose TableReference doesn't resolve — table never created, wrong dataset/project in the spec, or table deleted between pipeline construction and execution.","commonSituations":"Typos in the table spec string 'project:dataset.table'; environment-dependent table names (staging vs prod); race where an upstream stage was supposed to create the table before the read; temp/dynamic tables cleaned up too early.","solutions":["Verify the table exists: bq show 'project:dataset.table' with the same credentials/project","Fix the table spec string or TableReference (project, dataset, tableId)","Ensure any table-producing stage completes (or the table is created) before the read stage runs","Check the executing job's project — the table must exist in the project resolved by BigQueryOptions"],"exampleFix":"// before\nBigQueryIO.readTableRows().from(\"my-project:prod.events\"); // events doesn't exist\n// after\nBigQueryIO.readTableRows().from(\"my-project:prod.events_v2\");\n// or: bq mk --table my-project:prod.events schema.json","handlingStrategy":"validation","validationCode":"// before running the pipeline\ntry (BigQueryServices.DatasetService ds = bqServices.getDatasetService(options.as(BigQueryOptions.class))) {\n  if (ds.getTable(tableRef) == null) {\n    throw new IllegalStateException(\"Table missing: \" + BigQueryHelpers.toTableSpec(tableRef));\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Cannot start an export job since table\")) {\n    // verify/create table, fix spec, re-run\n  }\n}","preventionTips":["Verify table existence with bq show before launching jobs","Use exact project:dataset.table specs from config, not hand-typed strings","Ensure table-producing stages complete before read stages","Pin the executing project via --project to avoid cross-project lookups"],"tags":["bigquery","gcp","table-not-found","source"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}