{"record":{"id":"82928f40f3b6e315","repo":"apache/beam","slug":"duplicate-job-id-jobid","errorCode":null,"errorMessage":"Duplicate job id ${jobId}","messagePattern":"Duplicate job id (.+?)","errorType":"validation","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/FakeJobService.java","lineNumber":336,"sourceCode":"                      .setMessage(\n                          String.format(\n                              \"Job %s failed: %s\", job.job.getConfiguration(), e.toString())));\n          List<ResourceId> sourceFiles =\n              filesForLoadJobs.get(jobRef.getProjectId(), jobRef.getJobId());\n          if (sourceFiles != null) {\n            FileSystems.delete(sourceFiles);\n          }\n        }\n        return JSON_FACTORY.fromString(JSON_FACTORY.toString(job.job), Job.class);\n      }\n    } catch (IOException e) {\n      return null;\n    }\n  }\n\n  private void verifyUniqueJobId(String jobId) throws IOException {\n    if (allJobs.containsColumn(jobId)) {\n      throw new IOException(\"Duplicate job id \" + jobId);\n    }\n  }\n\n  private JobStatus runJob(Job job) throws InterruptedException, IOException {\n    if (job.getConfiguration().getLoad() != null) {\n      return runLoadJob(job.getJobReference(), job.getConfiguration().getLoad());\n    } else if (job.getConfiguration().getCopy() != null) {\n      return runCopyJob(job.getConfiguration().getCopy());\n    } else if (job.getConfiguration().getExtract() != null) {\n      return runExtractJob(job, job.getConfiguration().getExtract());\n    } else if (job.getConfiguration().getQuery() != null) {\n      return runQueryJob(job.getConfiguration().getQuery());\n    }\n    return new JobStatus().setState(\"DONE\");\n  }\n\n  private boolean validateDispositions(\n      Table table, CreateDisposition createDisposition, WriteDisposition writeDisposition)","sourceCodeStart":318,"sourceCodeEnd":354,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/testing/FakeJobService.java#L318-L354","documentation":"FakeJobService.verifyUniqueJobId enforces, like real BigQuery, that job ids are unique within the fake's job table. When startLoadJob, startExtractJob, or startCopyJob is called with a jobId already present in allJobs, it throws IOException(\"Duplicate job id \" + jobId). Tests that replay jobs or generate non-unique ids trip this.","triggerScenarios":"Calling startLoadJob/startExtractJob/startCopyJob with a job id already used in the same fake (job reference id reused across two starts), or re-running a test against a shared fake whose allJobs table still holds earlier jobs.","commonSituations":"Test retries that resubmit the same job reference without changing the id; deterministic id generation (e.g., constant strings) in test pipelines; a shared FakeJobService across test methods without cleanup.","solutions":["Generate a unique job id per submission (e.g., append a counter, UUID, or timestamp).","If idempotent resubmission is intended, catch the IOException or check job existence via getJob before starting.","Create a fresh FakeJobService (or clear allJobs) for each test to avoid cross-test id collisions.","Namespace job ids with the test name to guarantee uniqueness."],"exampleFix":"// before\nservice.startLoadJob(jobId, config);\nservice.startLoadJob(jobId, config2); // duplicate\n// after\nservice.startLoadJob(jobId + \"-1\", config);\nservice.startLoadJob(jobId + \"-2\", config2);","handlingStrategy":"validation","validationCode":"java.util.Set<String> usedJobIds = new java.util.HashSet<>();\nString jobId = baseId + \"-\" + java.util.UUID.randomUUID();\nif (!usedJobIds.add(jobId)) {\n  throw new IllegalStateException(\"Duplicate job id generated: \" + jobId);\n}","typeGuard":null,"tryCatchPattern":"try {\n  service.startLoadJob(jobRef, config);\n} catch (IOException e) {\n  if (e.getMessage().startsWith(\"Duplicate job id\")) {\n    // treat as already-submitted or regenerate id and retry once\n    jobRef = jobRef.toBuilder().setJobId(jobRef.getJobId() + \"-retry\").build();\n    service.startLoadJob(jobRef, config);\n  } else {\n    throw e;\n  }\n}","preventionTips":["Generate job ids with UUIDs or monotonic counters in test helpers.","Instantiate a fresh FakeJobService per test to keep allJobs empty.","Before starting, probe getJob to detect an existing id when resubmission is possible."],"tags":["java","bigquery","test-harness","duplicate-job-id","uniqueness"],"backgroundTag":"file-already-exists","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}