{"record":{"id":"373f5e9c0d05c39b","repo":"apache/beam","slug":"load-job-failed-with","errorCode":null,"errorMessage":"Load job {} failed with {}","messagePattern":"Load job (.+?) failed with (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java","lineNumber":534,"sourceCode":"    PendingJob retryJob =\n        new PendingJob(\n            // Function to load the data.\n            jobId -> {\n              JobReference jobRef =\n                  new JobReference()\n                      .setProjectId(projectId)\n                      .setJobId(jobId.getJobId())\n                      .setLocation(bqLocation);\n              LOG.info(\n                  \"Loading {} files into {} using job {}, job id iteration {}\",\n                  gcsUris.size(),\n                  ref,\n                  jobRef,\n                  jobId.getRetryIndex());\n              try {\n                jobService.startLoadJob(jobRef, loadConfig);\n              } catch (IOException | InterruptedException e) {\n                LOG.warn(\"Load job {} failed with {}\", jobRef, e.toString());\n                throw new RuntimeException(e);\n              }\n              return null;\n            },\n            // Function to poll the result of a load job.\n            jobId -> {\n              JobReference jobRef =\n                  new JobReference()\n                      .setProjectId(projectId)\n                      .setJobId(jobId.getJobId())\n                      .setLocation(bqLocation);\n              try {\n                return jobService.pollJob(jobRef, BatchLoads.LOAD_JOB_POLL_MAX_RETRIES);\n              } catch (InterruptedException e) {\n                throw new RuntimeException(e);\n              }\n            },\n            // Function to lookup a job.","sourceCodeStart":516,"sourceCodeEnd":552,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java#L516-L552","documentation":"A BigQuery load job (started via startLoadJob to bulk-load files into a BigQuery table) failed with an IOException or InterruptedException. The code logs the failure and rethrows it wrapped in a RuntimeException so the BigQueryIO retry logic can attempt the load again with a new retry index. This is part of WriteTables' batch-load path.","triggerScenarios":"The call jobService.startLoadJob(jobRef, loadConfig) throws IOException (BigQuery API error: quota exceeded, invalid schema/source format, table not found, permission denied) or InterruptedException (thread interrupted during API call).","commonSituations":"Load jobs failing due to malformed CSV/Avro/JSON source files, BigQuery rate limits, service account lacking bigquery.jobs.create, transient 500s from the BigQuery API, or pipeline cancellation interrupting the worker.","solutions":["Inspect the wrapped cause (e.getCause()) for the BigQuery API error details and fix the underlying issue (schema, format, permissions)","Check the service account has BigQuery Job User and Data Editor roles on the target dataset","Retry the pipeline; BigQueryIO automatically retries load jobs with an incremented retry index","Verify source URIs/files are readable and match the configured load format"],"exampleFix":"// before\nthrow new RuntimeException(e);\n// after\n// keep the wrap (needed for DoFn), but log the cause clearly:\nLOG.error(\"BigQuery load job {} failed\", jobRef, e.getCause());\nthrow new RuntimeException(\"BigQuery load job failed: \" + jobRef.getJobId(), e);","handlingStrategy":"retry","validationCode":"// pre-check before starting the load job\nthrowIf(!gcsUris.stream().allMatch(FileSystems::match), \"source URIs unreachable\");\nthrowIf(!svcAccountHasRole(\"roles/bigquery.jobUser\"), \"missing BigQuery Job User role\");","typeGuard":null,"tryCatchPattern":"try {\n  jobService.startLoadJob(jobRef, loadConfig);\n} catch (IOException e) {\n  if (isRetryable(e)) throw e; // let BigQueryIO retry logic handle it\n  throw new RuntimeException(\"non-retryable load failure: \" + jobRef.getJobId(), e);\n} catch (InterruptedException e) {\n  Thread.currentThread().interrupt();\n  throw new RuntimeException(\"load job interrupted\", e);\n}","preventionTips":["Validate source GCS files exist and match the declared format/schema before the load","Grant BigQuery Job User + Data Editor roles to the pipeline service account","Monitor BigQuery quota usage and set reasonable load job sizes","Keep source format flags (skip_leading_rows, encoding) consistent with actual files"],"tags":["gcp","bigquery","io","retryable"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}