{"record":{"id":"c656296d6ae6a339","repo":"apache/hadoop","slug":"object-s-already-exists","errorCode":null,"errorMessage":"Object %s already exists.","messagePattern":"Object (.+?) already exists\\.","errorType":"exception","errorClass":"FileAlreadyExistsException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorage.java","lineNumber":137,"sourceCode":"   * @param resourceId object for which generation info is requested\n   * @param overwrite  whether existing object should be overwritten\n   * @return the generation of the object\n   * @throws IOException if the object already exists and cannot be overwritten\n   */\n  private long getWriteGeneration(StorageResourceId resourceId, boolean overwrite)\n      throws IOException {\n    LOG.trace(\"getWriteGeneration({}, {})\", resourceId, overwrite);\n    GoogleCloudStorageItemInfo info = getItemInfo(resourceId);\n    if (!info.exists()) {\n      return 0L;\n    }\n    if (info.exists() && overwrite) {\n      long generation = info.getContentGeneration();\n      checkState(generation != 0, \"Generation should not be 0 for an existing item\");\n      return generation;\n    }\n\n    throw new FileAlreadyExistsException(String.format(\"Object %s already exists.\", resourceId));\n  }\n\n  void close() {\n    try {\n      storage.close();\n    } catch (Exception e) {\n      LOG.warn(\"Error occurred while closing the storage client\", e);\n    }\n  }\n\n  GoogleCloudStorageItemInfo getItemInfo(StorageResourceId resourceId) throws IOException {\n    LOG.trace(\"getItemInfo({})\", resourceId);\n\n    // Handle ROOT case first.\n    if (resourceId.isRoot()) {\n      return GoogleCloudStorageItemInfo.ROOT_INFO;\n    }\n    GoogleCloudStorageItemInfo itemInfo = null;","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorage.java#L119-L155","documentation":"getWriteGeneration(resourceId, overwrite) implements create-without-overwrite semantics using GCS object generations: for a non-existing object it returns 0, for an existing object with overwrite=true it returns the current generation to satisfy, and for an existing object with overwrite=false it throws FileAlreadyExistsException. The generationId is then used as a precondition on the write, so concurrent creators race safely and the loser gets this exception.","triggerScenarios":"Calling create() on a GCS path that already exists while overwrite is false (fs.create(path, false), or committers that create output objects exclusively); two writers targeting the same gs://bucket/object simultaneously; job retry re-creating the same task output.","commonSituations":"Re-running a job without cleaning previous _temporary/output directories; custom OutputCommitter or direct-storage writers using overwrite=false; concurrent tasks computing the same deterministic output path.","solutions":["Delete or move the existing object before re-creating it (fs.delete(path, false) when exists)","Write with overwrite=true if clobbering is acceptable: fs.create(path, true)","Use unique output paths per attempt (task attempt IDs) as the built-in committers do","On MapReduce/Spark, ensure the previous job's _temporary tree was cleaned before rerun"],"exampleFix":"// before\nFSDataOutputStream out = fs.create(path, false); // throws FileAlreadyExistsException\n\n// after\nif (fs.exists(path)) {\n  fs.delete(path, false);\n}\nFSDataOutputStream out = fs.create(path, false);","handlingStrategy":"try-catch","validationCode":"// Check before exclusive create\nif (fs.exists(path)) {\n  fs.delete(path, false); // or choose a unique output name\n}\nFSDataOutputStream out = fs.create(path, false);","typeGuard":"boolean isAlreadyExists(IOException e) {\n  return e instanceof FileAlreadyExistsException;\n}","tryCatchPattern":"try {\n  out = fs.create(path, false);\n} catch (FileAlreadyExistsException e) {\n  // another writer won the create race — pick a new name or verify and skip\n  path = uniqueAttemptPath(path);\n  out = fs.create(path, false);\n}","preventionTips":["Clean previous output/_temporary directories before job reruns","Use attempt-unique output paths as the standard committers do","Pass overwrite=true only when clobbering is semantically safe"],"tags":["gcs","hadoop-gcp","file-already-exists","overwrite","optimistic-concurrency"],"backgroundTag":"file-already-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}