apache/beam · error

Error extracting table. Note that external tables cannot be…

Error message

Error extracting table. Note that external tables cannot be exported: https://cloud.google.com/bigquery/docs/external-tables#external_table_limitations

What it means

In BigQuerySourceBase.executeExtract, if starting or polling the BigQuery extract job throws an IOException, Beam logs this warning as a breadcrumb before rethrowing. It exists because BigQuery's generic error messages can be misleading — one common root cause is attempting to export an external table, which BigQuery does not support.

Solutions

1. Check whether the source table is an external table in the BigQuery console; if so, copy it to a native table first (CREATE TABLE ... AS SELECT) and read that.
2. Inspect the chained IOException for the real BigQuery error message (permission denied, table not found, etc.).
3. Materialize the external table data into a native table or use STORAGE_API reads which don't require extract.
4. Retry if the underlying IOException was transient (network/quota).

Example fix

// before
BigQueryIO.read().from("project:dataset.external_table")
// after
// CREATE OR REPLACE TABLE dataset.native_copy AS SELECT * FROM dataset.external_table;
BigQueryIO.read().from("project:dataset.native_copy")

// Check table type before reading:
// SELECT table_type FROM <dataset>.INFORMATION_SCHEMA.TABLES WHERE table_name = '<table>';
// table_type 'EXTERNAL' means extract (and thus BigQueryIO classic read) will fail.

try { startExtractJob(...); } catch (IOException e) { if (isExternalTable(tableRef)) { throw new IllegalArgumentException("External table cannot be exported; materialize it first", e); } throw e; }

When it happens

Trigger: Executing an extract job (read via EXPORT / tempFiles) against a BigQuery table that is an external table (e.g. backed by GCS/Cloud SQL federated data), or any IOException during startExtractJob/pollJob.

Common situations: Reading from a BigQuery external table with BigQueryIO, which requires an extract-to-GCS step; mistakenly pointing BigQueryIO.read at an external table; transient API failures during extract that mask the external-table limitation.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• AvroRowWriter is not readable
• Could not create destination for extract job
• Extract job failed, status: .
• JsonRowWriter is not readable
• Load job failed with

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/797d43b1d0d113fa. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java:231

        new JobReference().setProjectId(executingProject).setLocation(bqLocation).setJobId(jobId);

    String destinationUri = BigQueryIO.getExtractDestinationUri(extractDestinationDir);
    JobConfigurationExtract extract =
        new JobConfigurationExtract()
            .setSourceTable(table)
            .setDestinationFormat("AVRO")
            .setUseAvroLogicalTypes(useAvroLogicalTypes)
            .setDestinationUris(ImmutableList.of(destinationUri));

    Job extractJob;
    try {
      LOG.info("Starting BigQuery extract job: {}", jobId);
      jobService.startExtractJob(jobRef, extract);
      extractJob = jobService.pollJob(jobRef, JOB_POLL_MAX_RETRIES);
    } catch (IOException exn) {
      // The error messages thrown in this case are generic and misleading, so leave this breadcrumb
      // in case it's the root cause.
      LOG.warn(
          "Error extracting table. Note that external tables cannot be exported: "
              + "https://cloud.google.com/bigquery/docs/external-tables#external_table_limitations",
          exn);
      throw exn;
    }
    if (BigQueryHelpers.parseStatus(extractJob) != Status.SUCCEEDED) {
      throw new IOException(
          String.format(
              "Extract job %s failed, status: %s.",
              extractJob.getJobReference().getJobId(),
              BigQueryHelpers.statusToPrettyString(extractJob.getStatus())));
    }

    LOG.info("BigQuery extract job completed: {}", jobId);

    return BigQueryIO.getExtractFilePaths(extractDestinationDir, extractJob);
  }

View on GitHub (pinned to 12126d8942)