{"record":{"id":"24400d47f57b4054","repo":"apache/beam","slug":"extract-job-s-failed-status-s","errorCode":null,"errorMessage":"Extract job %s failed, status: %s.","messagePattern":"Extract job (.+?) failed, status: (.+?)\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java","lineNumber":238,"sourceCode":"            .setUseAvroLogicalTypes(useAvroLogicalTypes)\n            .setDestinationUris(ImmutableList.of(destinationUri));\n\n    Job extractJob;\n    try {\n      LOG.info(\"Starting BigQuery extract job: {}\", jobId);\n      jobService.startExtractJob(jobRef, extract);\n      extractJob = jobService.pollJob(jobRef, JOB_POLL_MAX_RETRIES);\n    } catch (IOException exn) {\n      // The error messages thrown in this case are generic and misleading, so leave this breadcrumb\n      // in case it's the root cause.\n      LOG.warn(\n          \"Error extracting table. Note that external tables cannot be exported: \"\n              + \"https://cloud.google.com/bigquery/docs/external-tables#external_table_limitations\",\n          exn);\n      throw exn;\n    }\n    if (BigQueryHelpers.parseStatus(extractJob) != Status.SUCCEEDED) {\n      throw new IOException(\n          String.format(\n              \"Extract job %s failed, status: %s.\",\n              extractJob.getJobReference().getJobId(),\n              BigQueryHelpers.statusToPrettyString(extractJob.getStatus())));\n    }\n\n    LOG.info(\"BigQuery extract job completed: {}\", jobId);\n\n    return BigQueryIO.getExtractFilePaths(extractDestinationDir, extractJob);\n  }\n\n  List<BoundedSource<T>> createSources(\n      List<ResourceId> files, TableSchema schema, @Nullable List<MatchResult.Metadata> metadata)\n      throws IOException, InterruptedException {\n    String avroSchema = BigQueryAvroUtils.toGenericAvroSchema(schema).toString();\n\n    AvroSource.DatumReaderFactory<T> factory = readerFactory.apply(schema);\n","sourceCodeStart":220,"sourceCodeEnd":256,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java#L220-L256","documentation":"BigQuerySourceBase.executeExtract launches a BigQuery extract (export) job to dump the table to GCS files, then checks the job status. If BigQueryHelpers.parseStatus reports anything other than SUCCEEDED, the job's status is wrapped in an IOException and rethrown. This means the server-side export job itself failed (or has a failed/error state), not the client call.","triggerScenarios":"Calling BigQueryIO.read/transform that extracts a table to temporary GCS files when the extract job ends in a non-SUCCEEDED state (quota exceeded, destination bucket missing/permission-denied, unsupported source such as an external or materialized-view-like table, invalid destination format/URI, transient BigQuery backend error).","commonSituations":"Exporting a table whose data exceeds the 1GB-per-file or multi-wildcard limits; GCS temp location bucket doesn't exist or the service account lacks write permission; attempting to export external tables; BigQuery transient job failures during heavy load.","solutions":["Read the pretty-printed job status in the message to get the exact BigQuery error (e.g. accessDenied, invalid, rateLimitExceeded) and fix that underlying cause first","Verify the --tempLocation / tempWriteGcsBucket GCS bucket exists and the project's BigQuery service account has OBJECT_WRITER permission on it","Confirm the source is a normal (non-external) table; external tables cannot be exported","Retry the pipeline if the status indicates a transient BigQuery error; consider enabling BigQuery job retry options"],"exampleFix":"// before\npipeline.apply(\"Read\", BigQueryIO.readTableRows().from(\"proj:ds.tbl\").withMethod(EXTRACT));\n// after\n// ensure a valid, writable temp bucket and non-external table:\nBigQueryIO.readTableRows()\n    .from(\"proj:ds.tbl\")\n    .withMethod(Method.EXTRACT)\n    .withTempWriteGcsBucket(\"my-writable-bucket\") // must exist & allow BigQuery export\n    .withKmsKey(...) // only if required by bucket policy","handlingStrategy":"try-catch","validationCode":"// Pre-check: bucket exists & service account can write\nStorage storage = StorageOptions.getDefaultInstance().getService();\nif (storage.get(tempBucket) == null) throw new IllegalArgumentException(\"tempLocation bucket missing\");\n// and confirm the table is not external:\n// bq show --format=json project:dataset.table | jq -r '.type' -> must be TABLE/VIEW exportable","typeGuard":"boolean isExportableTable(Table t) {\n  return t != null && !\"EXTERNAL\".equals(t.getType());\n}","tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Extract job \")) {\n    // parse BigQueryHelpers.statusToPrettyString payload for error.status and handle specific causes\n  }\n  throw e;\n}","preventionTips":["Ensure --tempLocation points to an existing bucket the BigQuery service account can write to","Never use EXTRACT method on external tables","Monitor BigQuery job history (bq ls -j) for the extract job to get the exact failure reason","Retry transient failures with backoff (RateLimitExceeded, backend errors)"],"tags":["bigquery","gcs","job-failure","io"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}