{"record":{"id":"26bdc7d7962ffba9","repo":"apache/beam","slug":"unable-to-rename-resource-s-to-s-as-destination-already","errorCode":null,"errorMessage":"Unable to rename resource %s to %s as destination already exists and couldn't be deleted.","messagePattern":"Unable to rename resource (.+?) to (.+?) as destination already exists and couldn't be deleted\\.","errorType":"exception","errorClass":"FileAlreadyExistsException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java","lineNumber":285,"sourceCode":"      boolean success = fs.rename(srcPath, destPath);\n\n      // If the failure was due to the file already existing, delete and retry (BEAM-5036).\n      // This should be the exceptional case, so handle here rather than incur the overhead of\n      // testing first\n      if (!success && fs.exists(srcPath) && fs.exists(destPath)) {\n        LOG.debug(LOG_DELETING_EXISTING_FILE, Path.getPathWithoutSchemeAndAuthority(destPath));\n        fs.delete(destPath, false); // not recursive\n        success = fs.rename(srcPath, destPath);\n      }\n\n      if (!success) {\n        if (!fs.exists(srcPath)) {\n          throw new FileNotFoundException(\n              String.format(\n                  \"Unable to rename resource %s to %s as source not found.\", srcPath, destPath));\n\n        } else if (fs.exists(destPath)) {\n          throw new FileAlreadyExistsException(\n              String.format(\n                  \"Unable to rename resource %s to %s as destination already exists and couldn't be deleted.\",\n                  srcPath, destPath));\n\n        } else {\n          throw new IOException(\n              String.format(\n                  \"Unable to rename resource %s to %s. No further information provided by underlying filesystem.\",\n                  srcPath, destPath));\n        }\n      }\n    }\n  }\n\n  /** Ensures that the target directory exists for the given filePath. */\n  private void mkdirs(Path filePath) throws IOException {\n    final org.apache.hadoop.fs.FileSystem fs = filePath.getFileSystem(configuration);\n    final Path targetDirectory = filePath.getParent();","sourceCodeStart":267,"sourceCodeEnd":303,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java#L267-L303","documentation":"If fs.rename() returns false and the destination still exists after Beam attempted fs.delete(destPath, false), rename() throws FileAlreadyExistsException with this message. The Hadoop rename refuses to overwrite an existing destination and Beam's non-recursive delete failed to clear it.","triggerScenarios":"Calling HadoopFileSystem.rename() where the destination path already exists and the pre-emptive non-recursive delete of the destination fails (permissions, dest is a non-empty directory, etc.).","commonSituations":"Re-running a pipeline whose previous run left output files at the destination; renaming a file onto an existing directory; delete blocked by permissions or a non-empty dir.","solutions":["Delete or move the existing destination before renaming.","Ensure the caller has delete permission on the destination path.","For directories, delete recursively or choose a fresh destination path."],"exampleFix":"// before\nfileSystem.rename(ImmutableList.of(src), ImmutableList.of(dest));\n// after\ntry {\n  fileSystem.delete(ImmutableList.of(dest));\n} catch (FileNotFoundException ignored) {\n}\nfileSystem.rename(ImmutableList.of(src), ImmutableList.of(dest));","handlingStrategy":"validation","validationCode":"if (fileSystem.match(dest.toString()).status() == MatchResult.Status.OK) {\n  fileSystem.delete(ImmutableList.of(dest));\n}","typeGuard":null,"tryCatchPattern":"try {\n  fileSystem.rename(srcs, dests);\n} catch (FileAlreadyExistsException e) {\n  fileSystem.delete(ImmutableList.of(dest));\n  fileSystem.rename(srcs, dests);\n}","preventionTips":["Clean or version output destinations before renames","Ensure the job user has delete permission on the destination","Use unique, timestamped destination paths on reruns"],"tags":["hadoop","hdfs","already-exists","beam"],"backgroundTag":"file-already-exists","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}