apache/beam · error · IOException

Cannot start an export job since table

Error message

Cannot start an export job since table %s does not exist

What it means

BigQuerySourceBase.extractFiles starts an export (extract) job to dump the source table to GCS, but first fetches the table via DatasetService.getTable; if the table is null (does not exist), it throws this IOException with the table spec. The read cannot proceed without a table to export.

Solutions

1. Verify the table exists: bq show 'project:dataset.table' with the same credentials/project
2. Fix the table spec string or TableReference (project, dataset, tableId)
3. Ensure any table-producing stage completes (or the table is created) before the read stage runs
4. Check the executing job's project — the table must exist in the project resolved by BigQueryOptions

Example fix

// before
BigQueryIO.readTableRows().from("my-project:prod.events"); // events doesn't exist
// after
BigQueryIO.readTableRows().from("my-project:prod.events_v2");
// or: bq mk --table my-project:prod.events schema.json

// before running the pipeline
try (BigQueryServices.DatasetService ds = bqServices.getDatasetService(options.as(BigQueryOptions.class))) {
  if (ds.getTable(tableRef) == null) {
    throw new IllegalStateException("Table missing: " + BigQueryHelpers.toTableSpec(tableRef));
  }
}

try {
  pipeline.run().waitUntilFinish();
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Cannot start an export job since table")) {
    // verify/create table, fix spec, re-run
  }
}

When it happens

Trigger: Reading from a BigQuery table (BigQueryIO.read / TableSource) whose TableReference doesn't resolve — table never created, wrong dataset/project in the spec, or table deleted between pipeline construction and execution.

Common situations: Typos in the table spec string 'project:dataset.table'; environment-dependent table names (staging vs prod); race where an upstream stage was supposed to create the table before the read; temp/dynamic tables cleaned up too early.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Insert failed:
• Append to stream failed with invalid offset of
• Append to stream failed with Status Code . The stream may…
• BigQuery source must be split before being read
• BigQuery table " + tableReference + " not found. If you…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/660d45594086fe9d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java:118

    }

    public ExtractResult(
        TableSchema schema,
        List<ResourceId> extractedFiles,
        @Nullable List<MatchResult.Metadata> metadata) {
      this.schema = schema;
      this.extractedFiles = extractedFiles;
      this.metadata = metadata;
    }
  }

  protected ExtractResult extractFiles(PipelineOptions options) throws Exception {
    BigQueryOptions bqOptions = options.as(BigQueryOptions.class);
    TableReference tableToExtract = getTableToExtract(bqOptions);
    try (BigQueryServices.DatasetService datasetService = bqServices.getDatasetService(bqOptions)) {
      Table table = datasetService.getTable(tableToExtract);
      if (table == null) {
        throw new IOException(
            String.format(
                "Cannot start an export job since table %s does not exist",
                BigQueryHelpers.toTableSpec(tableToExtract)));
      }
      // emit this table ID as a lineage source
      Lineage.getSources()
          .add("bigquery", BigQueryHelpers.dataCatalogSegments(tableToExtract, bqOptions));

      TableSchema schema = table.getSchema();
      JobService jobService = bqServices.getJobService(bqOptions);
      String extractJobId =
          BigQueryResourceNaming.createJobIdPrefix(options.getJobName(), stepUuid, JobType.EXPORT);
      final String extractDestinationDir =
          resolveTempLocation(bqOptions.getTempLocation(), "BigQueryExtractTemp", stepUuid);
      String bqLocation =
          BigQueryHelpers.getDatasetLocation(
              datasetService, tableToExtract.getProjectId(), tableToExtract.getDatasetId());
      String bqProjectId =

View on GitHub (pinned to 12126d8942)