{"record":{"id":"51f8051a5a5541b6","repo":"apache/hadoop","slug":"destination-is-an-existed-file","errorCode":null,"errorMessage":"destination is an existed file","messagePattern":"destination is an existed file","errorType":"exception","errorClass":"RenameFailedException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java","lineNumber":144,"sourceCode":"        if (dstStatus.isDirectory()) {\n          throw new RenameFailedException(src, dst,\n              \"new destination is an existed directory\")\n              .withExitCode(false);\n        } else {\n          throw new RenameFailedException(src, dst,\n              \"new destination is an existed file\")\n              .withExitCode(false);\n        }\n      } else {\n\n        if (srcKey.equals(dstKey)) {\n          LOG.warn(\n              \"rename: src and dest refer to the same file or\"\n                  + \" directory: {}\",\n              dst);\n          return true;\n        } else {\n          throw new RenameFailedException(src, dst,\n              \"destination is an existed file\")\n              .withExitCode(false);\n        }\n      }\n    } catch (FileNotFoundException e) {\n      LOG.debug(\"rename: destination path {} not found\", dst);\n\n      // Parent must exist\n      checkDestinationParent(owner, src, dst);\n    }\n\n    if (dstKey.startsWith(srcKey)\n        && dstKey.charAt(srcKey.length()) == Path.SEPARATOR_CHAR) {\n      LOG.error(\"rename: dest [{}] cannot be a descendant of src [{}]\",\n          dst, src);\n      return false;\n    }\n","sourceCodeStart":126,"sourceCodeEnd":162,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java#L126-L162","documentation":"When the destination of a rename exists and is a file (not a directory), Hadoop semantics forbid silently overwriting it: if the source and destination keys differ, renameBasedOnObject throws RenameFailedException('destination is an existed file').withExitCode(false). Only the exact src==dst case is tolerated with a warning and a true return.","triggerScenarios":"fs.rename('/a/tmp-file', '/b/final-file') where /b/final-file is an existing different file; atomic commit patterns (write temp, rename to final) re-run when the final file already exists; two writers committing to the same final path.","commonSituations":"Job retry after a crash that happened after commit but before acknowledgment; output committers that assume rename-over-file works as on a local filesystem; multiple pipelines writing the same output key.","solutions":["Delete the existing destination file first when overwrite is intended: fs.delete(dst, false)","Make the commit idempotent: if the destination exists and the source is gone, treat the commit as done","Give each run a unique output path and let a downstream step pick the latest"],"exampleFix":"// before\nfs.rename(tmpPath, finalPath); // fails: destination is an existed file\n\n// after\nif (fs.exists(finalPath)) {\n  fs.delete(finalPath, false);\n}\nfs.rename(tmpPath, finalPath);","handlingStrategy":"validation","validationCode":"if (fs.exists(dst)) {\n  if (fs.getFileStatus(dst).isFile()) {\n    fs.delete(dst, false); // overwrite intended\n  } else {\n    throw new IOException(\"Destination exists and is not a file: \" + dst);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  fs.rename(tmpPath, finalPath);\n} catch (RenameFailedException e) {\n  if (e.getMessage().contains(\"destination is an existed file\")) {\n    // prior run committed already: delete or skip, then retry\n  } else {\n    throw e;\n  }\n}","preventionTips":["Never assume rename overwrites a file on object storage","Before a temp-to-final rename, check the final path and reconcile","Treat 'source gone + destination present' as a completed commit"],"tags":["hadoop","obs","rename","destination-exists","overwrite"],"backgroundTag":"rename-destination-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}