{"record":{"id":"20062cd53ffcf5db","repo":"apache/hadoop","slug":"source-src-and-destination-dst-must-both-be-di","errorCode":null,"errorMessage":"Source {src} and destination {dst} must both be directories","messagePattern":"Source (.+?) and destination (.+?) must both be directories","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java","lineNumber":872,"sourceCode":"\n    FileStatus dstStatus;\n    try {\n      dstStatus = getFileLinkStatus(dst);\n    } catch (IOException e) {\n      dstStatus = null;\n    }\n    if (dstStatus != null) {\n      if (dst.equals(src)) {\n        throw new FileAlreadyExistsException(\n            \"The source \"+src+\" and destination \"+dst+\" are the same\");\n      }\n      if (srcStatus.isSymlink() && dst.equals(srcStatus.getSymlink())) {\n        throw new FileAlreadyExistsException(\n            \"Cannot rename symlink \"+src+\" to its target \"+dst);\n      }\n      // It's OK to rename a file to a symlink and vice versa\n      if (srcStatus.isDirectory() != dstStatus.isDirectory()) {\n        throw new IOException(\"Source \" + src + \" and destination \" + dst\n            + \" must both be directories\");\n      }\n      if (!overwrite) {\n        throw new FileAlreadyExistsException(\"Rename destination \" + dst\n            + \" already exists.\");\n      }\n      // Delete the destination that is a file or an empty directory\n      if (dstStatus.isDirectory()) {\n        RemoteIterator<FileStatus> list = listStatusIterator(dst);\n        if (list != null && list.hasNext()) {\n          throw new IOException(\n              \"Rename cannot overwrite non empty destination directory \" + dst);\n        }\n      }\n      delete(dst, false);\n    } else {\n      final Path parent = dst.getParent();\n      final FileStatus parentStatus = getFileStatus(parent);","sourceCodeStart":854,"sourceCodeEnd":890,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java#L854-L890","documentation":"During overwrite rename, if the destination exists but source and destination disagree on isDirectory(), AbstractFileSystem.renameInternal throws this plain IOException: you may only overwrite a file with a file or a directory with a directory. Note the source comment: renaming a file to a symlink (or vice versa) is allowed because the link's own status is not a directory, so a directory-vs-symlink pair also lands here.","triggerScenarios":"fc.rename(srcDir, dstFile, Rename.OVERWRITE) or fc.rename(srcFile, dstDir, Rename.OVERWRITE) where dst exists; also renaming a directory onto an existing symlink (the symlink's own FileStatus is not a directory).","commonSituations":"Job reruns where a previous run left a _SUCCESS marker file at a path now expected to be a directory (or the reverse); committers switching output layout between file and directory; leftover artifacts from an older pipeline schema occupying the destination name.","solutions":["Delete the mismatched destination first (fc.delete(dst, true)) and then perform the overwrite rename","Choose a fresh destination path so a type collision is impossible","Normalize the pipeline so a given output path is always the same type across runs"],"exampleFix":"// before\nfc.rename(srcDir, dst, Options.Rename.OVERWRITE); // dst is a file -> IOException\n\n// after\nif (fc.util().exists(dst)\n    && fc.getFileStatus(src).isDirectory() != fc.getFileStatus(dst).isDirectory()) {\n  fc.delete(dst, true);\n}\nfc.rename(srcDir, dst, Options.Rename.OVERWRITE);","handlingStrategy":"validation","validationCode":"if (fc.util().exists(dst)\n    && fc.getFileStatus(src).isDirectory()\n        != fc.getFileStatus(dst).isDirectory()) {\n  fc.delete(dst, true); // or choose a different dst\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (e.getMessage().startsWith(\"Source \")) { /* type conflict: delete dst or retarget */ } else throw e; }","preventionTips":["Check isDirectory() of both ends before every OVERWRITE rename","Keep each output path a stable type across all pipeline runs","Treat a leftover file where a directory is expected as stale output and clean it in a pre-step"],"tags":["rename","overwrite","type-mismatch","filecontext","hadoop-fs"],"backgroundTag":"rename-destination-invalid","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}