apache/beam · critical · RuntimeException

Failed to create job with prefix

Error message

Failed to create job with prefix %s, reached max retries: %d, last failed job: %s.

What it means

Thrown by BigQueryHelpers' job-running retry loop (runJob) after the maximum number of attempts to create a BigQuery job has been exhausted. Every attempt failed (quota, transient API errors, invalid job config), so the library logs the last failed job attempt and wraps the failure in a RuntimeException including the job ID prefix, retry count, and the last job's pretty-printed description.

Solutions

1. Read the logged last failed job (jobToPrettyString output) to find the underlying BigQuery API error (e.g. quota, access denied).
2. Fix the root cause: request higher quota, grant the service account BigQuery Job User / Data Editor roles, or correct the job configuration.
3. Increase maxRetries or add backoff if failures are transient; re-run the pipeline once the service is healthy.

Example fix

// before
// pipeline fails: 'Failed to create job with prefix beam_job_..., reached max retries: 10'
// after
// grant roles/bigquery.jobUser to the pipeline service account and check quota in Cloud Console,
// then re-run the pipeline.

try {
  pipeline.run().waitUntilFinish();
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("reached max retries")) {
    // inspect cause / logged last failed job for the BigQuery API error before retrying
  }
}

When it happens

Trigger: BigQueryIO load/copy/export operations invoking DryRunJobLoadConfiguration/BigQueryJobCreateConfiguration-style runJob() where the BigQuery jobs.insert API call fails on every retry up to maxRetries.

Common situations: BigQuery quota exhausted or rate-limited (403 rateLimitExceeded), persistent permission errors on the project, invalid job configuration, or a prolonged BigQuery service outage while running a Beam pipeline write/load.

Related errors

• A function must be provided to convert the input type into…
• BigQuery %1$s not found for table "%2$s" . Please create…
• BigQuery data contained value
• BigQuery temp location expected a valid 'gs://' path, but…
• BigQuery temp location expected a valid 'gs://' path, but…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6334870c9116e095. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java:217

        SerializableFunction<RetryJobId, Job> pollJob,
        SerializableFunction<RetryJobId, Job> lookupJob,
        int maxRetries,
        String jobIdPrefix) {
      this.executeJob = executeJob;
      this.pollJob = pollJob;
      this.lookupJob = lookupJob;
      this.maxRetries = maxRetries;
      this.currentAttempt = 0;
      currentJobId = new RetryJobId(jobIdPrefix, 0);
      this.started = false;
    }

    // Run the job.
    void runJob() throws IOException {
      ++currentAttempt;
      if (!shouldRetry()) {
        logBigQueryError(lastJobAttempted);
        throw new RuntimeException(
            String.format(
                "Failed to create job with prefix %s, "
                    + "reached max retries: %d, last failed job: %s.",
                currentJobId.getJobIdPrefix(),
                maxRetries,
                BigQueryHelpers.jobToPrettyString(lastJobAttempted)));
      }

      try {
        this.started = false;
        executeJob.apply(currentJobId);
      } catch (RuntimeException e) {
        LOG.warn("Job {} failed.", currentJobId.getJobId(), e);
        // It's possible that the job actually made it to BQ even though we got a failure here.
        // For example, the response from BQ may have timed out returning. getRetryJobId will
        // return the correct job id to use on retry, or a job id to continue polling (if it turns
        // out that the job has not actually failed yet).
        RetryJobIdResult result = getRetryJobId(currentJobId, lookupJob);

View on GitHub (pinned to 12126d8942)