apache/hadoop · error · IOException

Unable to rename " + src + " to " + target

Error message

Unable to rename " + src + " to " + target

What it means

The terminal failure of replaceFile(File src, File target): after the initial src.renameTo(target) fails and the retry loop (up to 5 delete attempts of target with 1s sleeps) finishes, a second renameTo attempt that still fails throws IOException("Unable to rename <src> to <target>"). On Windows this is the documented behavior when the target is open for reading/writing (see the comment block in the source); on POSIX it means cross-filesystem rename, missing permissions, or a dangling directory target.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java:1583

   * @exception IOException If this operation fails
   */
  public static void replaceFile(File src, File target) throws IOException {
    /* renameTo() has two limitations on Windows platform.
     * src.renameTo(target) fails if
     * 1) If target already exists OR
     * 2) If target is already open for reading/writing.
     */
    if (!src.renameTo(target)) {
      int retries = 5;
      while (target.exists() && !target.delete() && retries-- >= 0) {
        try {
          Thread.sleep(1000);
        } catch (InterruptedException e) {
          throw new IOException("replaceFile interrupted.");
        }
      }
      if (!src.renameTo(target)) {
        throw new IOException("Unable to rename " + src +
                              " to " + target);
      }
    }
  }

  /**
   * A wrapper for {@link File#listFiles()}. This java.io API returns null
   * when a dir is not a directory or for any I/O error. Instead of having
   * null check everywhere File#listFiles() is used, we will add utility API
   * to get around this problem. For the majority of cases where we prefer
   * an IOException to be thrown.
   * @param dir directory for which listing should be performed
   * @return list of files or empty list
   * @exception IOException for invalid directory or for a bad disk.
   */
  public static File[] listFiles(File dir) throws IOException {
    File[] files = dir.listFiles();
    if(files == null) {

View on GitHub (pinned to 2add963021)

Solutions

  1. Close all handles on src and target in YOUR process before renaming; scan for leaked streams (try-with-resources)
  2. On Windows, exclude the directory from antivirus scanning or retry with backoff when external lockers (AV, indexer) are the culprit
  3. Ensure src and target are on the same filesystem/device; otherwise copy-then-delete instead of rename
  4. As a last resort, copy src to a unique temp file next to target and atomically Files.move it with ATOMIC_MOVE after deleting target

Example fix

// before
FileUtil.replaceFile(tmpFile, finalFile);
// IOException: Unable to rename tmpFile to finalFile

// after: guaranteed-closed handles + copy fallback across filesystems
try (InputStream in = Files.newInputStream(tmpFile.toPath());
     OutputStream out = Files.newOutputStream(finalTmp.toPath())) {
  in.transferTo(out);
}
Files.deleteIfExists(finalFile.toPath());
Files.move(finalTmp.toPath(), finalFile.toPath(),
    StandardCopyOption.ATOMIC_MOVE);
Defensive patterns

Strategy: retry

Validate before calling

// cheap pre-flight: same filesystem, writable target parent, target closeable
if (!src.getParentFile().equals(target.getParentFile())
    && !src.toPath().toRealPath().startsWith(target.getParentFile().toPath())) {
  // cross-device risk: copy+delete instead of rename
}
if (!target.getParentFile().canWrite()) throw new IOException("Target parent not writable");

Try / catch

for (int attempt = 0; attempt < 3; attempt++) {
  try {
    FileUtil.replaceFile(src, target);
    break;
  } catch (IOException e) {
    if (!e.getMessage().contains("Unable to rename")) throw e;
    closeLeakedHandles(target); // then wait out transient Windows locks (AV scan)
  }
}

Prevention

When it happens

Trigger: replaceFile on Windows with target held open by any process (antivirus scanners, indexers, your own unclosed streams); rename across mount points/filesystems; src no longer exists by the time of the second rename; target is a non-empty directory.

Common situations: LocalFileSystem rename during commit on Windows nodes; AV/backup software briefly locking files; renaming between /tmp and a data volume (different devices); leftover open FSDataInputStream from a read cache.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0826d595a332e368. Report an issue: GitHub.