{"record":{"id":"6334870c9116e095","repo":"apache/beam","slug":"failed-to-create-job-with-prefix-s-reached-max-retries-d","errorCode":null,"errorMessage":"Failed to create job with prefix %s, reached max retries: %d, last failed job: %s.","messagePattern":"Failed to create job with prefix (.+?), reached max retries: (.+?), last failed job: (.+?)\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java","lineNumber":217,"sourceCode":"        SerializableFunction<RetryJobId, Job> pollJob,\n        SerializableFunction<RetryJobId, Job> lookupJob,\n        int maxRetries,\n        String jobIdPrefix) {\n      this.executeJob = executeJob;\n      this.pollJob = pollJob;\n      this.lookupJob = lookupJob;\n      this.maxRetries = maxRetries;\n      this.currentAttempt = 0;\n      currentJobId = new RetryJobId(jobIdPrefix, 0);\n      this.started = false;\n    }\n\n    // Run the job.\n    void runJob() throws IOException {\n      ++currentAttempt;\n      if (!shouldRetry()) {\n        logBigQueryError(lastJobAttempted);\n        throw new RuntimeException(\n            String.format(\n                \"Failed to create job with prefix %s, \"\n                    + \"reached max retries: %d, last failed job: %s.\",\n                currentJobId.getJobIdPrefix(),\n                maxRetries,\n                BigQueryHelpers.jobToPrettyString(lastJobAttempted)));\n      }\n\n      try {\n        this.started = false;\n        executeJob.apply(currentJobId);\n      } catch (RuntimeException e) {\n        LOG.warn(\"Job {} failed.\", currentJobId.getJobId(), e);\n        // It's possible that the job actually made it to BQ even though we got a failure here.\n        // For example, the response from BQ may have timed out returning. getRetryJobId will\n        // return the correct job id to use on retry, or a job id to continue polling (if it turns\n        // out that the job has not actually failed yet).\n        RetryJobIdResult result = getRetryJobId(currentJobId, lookupJob);","sourceCodeStart":199,"sourceCodeEnd":235,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java#L199-L235","documentation":"Thrown by BigQueryHelpers' job-running retry loop (runJob) after the maximum number of attempts to create a BigQuery job has been exhausted. Every attempt failed (quota, transient API errors, invalid job config), so the library logs the last failed job attempt and wraps the failure in a RuntimeException including the job ID prefix, retry count, and the last job's pretty-printed description.","triggerScenarios":"BigQueryIO load/copy/export operations invoking DryRunJobLoadConfiguration/BigQueryJobCreateConfiguration-style runJob() where the BigQuery jobs.insert API call fails on every retry up to maxRetries.","commonSituations":"BigQuery quota exhausted or rate-limited (403 rateLimitExceeded), persistent permission errors on the project, invalid job configuration, or a prolonged BigQuery service outage while running a Beam pipeline write/load.","solutions":["Read the logged last failed job (jobToPrettyString output) to find the underlying BigQuery API error (e.g. quota, access denied).","Fix the root cause: request higher quota, grant the service account BigQuery Job User / Data Editor roles, or correct the job configuration.","Increase maxRetries or add backoff if failures are transient; re-run the pipeline once the service is healthy."],"exampleFix":"// before\n// pipeline fails: 'Failed to create job with prefix beam_job_..., reached max retries: 10'\n// after\n// grant roles/bigquery.jobUser to the pipeline service account and check quota in Cloud Console,\n// then re-run the pipeline.","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"reached max retries\")) {\n    // inspect cause / logged last failed job for the BigQuery API error before retrying\n  }\n}","preventionTips":["Pre-check BigQuery quota and rate limits before large pipeline runs.","Grant roles/bigquery.jobUser and roles/bigquery.dataEditor to the pipeline service account.","Monitor GCP status for BigQuery incidents before submitting long-running jobs.","Set retries/backoff appropriately for transient errors in the job creation path."],"tags":["java","bigquery","retry-exhausted","gcp-api"],"backgroundTag":"retry-exhausted","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}