{"record":{"id":"797d43b1d0d113fa","repo":"apache/beam","slug":"error-extracting-table-note-that-external-tables-cannot-be","errorCode":null,"errorMessage":"Error extracting table. Note that external tables cannot be exported: https://cloud.google.com/bigquery/docs/external-tables#external_table_limitations","messagePattern":"Error extracting table\\. Note that external tables cannot be exported: https://cloud\\.google\\.com/bigquery/docs/external-tables#external_table_limitations","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java","lineNumber":231,"sourceCode":"        new JobReference().setProjectId(executingProject).setLocation(bqLocation).setJobId(jobId);\n\n    String destinationUri = BigQueryIO.getExtractDestinationUri(extractDestinationDir);\n    JobConfigurationExtract extract =\n        new JobConfigurationExtract()\n            .setSourceTable(table)\n            .setDestinationFormat(\"AVRO\")\n            .setUseAvroLogicalTypes(useAvroLogicalTypes)\n            .setDestinationUris(ImmutableList.of(destinationUri));\n\n    Job extractJob;\n    try {\n      LOG.info(\"Starting BigQuery extract job: {}\", jobId);\n      jobService.startExtractJob(jobRef, extract);\n      extractJob = jobService.pollJob(jobRef, JOB_POLL_MAX_RETRIES);\n    } catch (IOException exn) {\n      // The error messages thrown in this case are generic and misleading, so leave this breadcrumb\n      // in case it's the root cause.\n      LOG.warn(\n          \"Error extracting table. Note that external tables cannot be exported: \"\n              + \"https://cloud.google.com/bigquery/docs/external-tables#external_table_limitations\",\n          exn);\n      throw exn;\n    }\n    if (BigQueryHelpers.parseStatus(extractJob) != Status.SUCCEEDED) {\n      throw new IOException(\n          String.format(\n              \"Extract job %s failed, status: %s.\",\n              extractJob.getJobReference().getJobId(),\n              BigQueryHelpers.statusToPrettyString(extractJob.getStatus())));\n    }\n\n    LOG.info(\"BigQuery extract job completed: {}\", jobId);\n\n    return BigQueryIO.getExtractFilePaths(extractDestinationDir, extractJob);\n  }\n","sourceCodeStart":213,"sourceCodeEnd":249,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java#L213-L249","documentation":"In BigQuerySourceBase.executeExtract, if starting or polling the BigQuery extract job throws an IOException, Beam logs this warning as a breadcrumb before rethrowing. It exists because BigQuery's generic error messages can be misleading — one common root cause is attempting to export an external table, which BigQuery does not support.","triggerScenarios":"Executing an extract job (read via EXPORT / tempFiles) against a BigQuery table that is an external table (e.g. backed by GCS/Cloud SQL federated data), or any IOException during startExtractJob/pollJob.","commonSituations":"Reading from a BigQuery external table with BigQueryIO, which requires an extract-to-GCS step; mistakenly pointing BigQueryIO.read at an external table; transient API failures during extract that mask the external-table limitation.","solutions":["Check whether the source table is an external table in the BigQuery console; if so, copy it to a native table first (CREATE TABLE ... AS SELECT) and read that.","Inspect the chained IOException for the real BigQuery error message (permission denied, table not found, etc.).","Materialize the external table data into a native table or use STORAGE_API reads which don't require extract.","Retry if the underlying IOException was transient (network/quota)."],"exampleFix":"// before\nBigQueryIO.read().from(\"project:dataset.external_table\")\n// after\n// CREATE OR REPLACE TABLE dataset.native_copy AS SELECT * FROM dataset.external_table;\nBigQueryIO.read().from(\"project:dataset.native_copy\")","handlingStrategy":"validation","validationCode":"// Check table type before reading:\n// SELECT table_type FROM <dataset>.INFORMATION_SCHEMA.TABLES WHERE table_name = '<table>';\n// table_type 'EXTERNAL' means extract (and thus BigQueryIO classic read) will fail.","typeGuard":null,"tryCatchPattern":"try { startExtractJob(...); } catch (IOException e) { if (isExternalTable(tableRef)) { throw new IllegalArgumentException(\"External table cannot be exported; materialize it first\", e); } throw e; }","preventionTips":["Never point BigQueryIO.read at EXTERNAL tables without materializing first.","Check INFORMATION_SCHEMA.TABLES.table_type during pipeline setup.","Materialize external tables via CTAS before extract-based reads."],"tags":["bigquery","extract","external-table","io"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}