apache/hadoop · error · IOException

Failed to rename {from} to {to} due to failure in native ren

Error message

Failed to rename {from} to {to} due to failure in native rename. {e}

What it means

Storage.rename(File,File) delegates to NativeIO.renameTo; when the native library is loaded, renameTo0 throws NativeIOException carrying the underlying errno (EACCES, EINVAL, EPERM, ...) or Windows error, which Storage.rename rewraps with both canonical paths. It is the atomic-replace primitive used when moving storage metadata files (e.g. temporary VERSION/checkpoint files) into place. When native code is not loaded, NativeIO.renameTo instead throws a plain IOException with a less descriptive message.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java:1288

       */
      props.store(out, null);
      /*
       * Now the new fields are flushed to the head of the file, but file
       * length can still be larger then required and therefore the file can
       * contain whole or corrupted fields from its old contents in the end.
       * If server is interrupted here and restarted later these extra fields
       * either should not effect server behavior or should be handled
       * by the server correctly.
       */
      file.setLength(out.getChannel().position());
    }
  }

  public static void rename(File from, File to) throws IOException {
    try {
      NativeIO.renameTo(from, to);
    } catch (NativeIOException e) {
      throw new IOException("Failed to rename " + from.getCanonicalPath()
        + " to " + to.getCanonicalPath() + " due to failure in native rename. "
        + e.toString());
    }
  }

  /**
   * Copies a file (usually large) to a new location using native unbuffered IO.
   * <p>
   * This method copies the contents of the specified source file
   * to the specified destination file using OS specific unbuffered IO.
   * The goal is to avoid churning the file system buffer cache when copying
   * large files.
   *
   * We can't use FileUtils#copyFile from apache-commons-io because it
   * is a buffered IO based on FileChannel#transferFrom, which uses MmapByteBuffer
   * internally.
   *
   * The directory holding the destination file is created if it does not exist.

View on GitHub (pinned to 2add963021)

Solutions

Read the NativeIOException detail in the message - it carries the errno; fix accordingly (errno 13/EACCES = permissions)
chown/chmod the storage directory so the daemon user owns it: chown -R hdfs:hdfs <storage-dir>
Remove the stale destination file if one exists
Confirm src and dst live on the same filesystem and it is mounted read-write

Example fix

// before
Storage.rename(tmpFile, destFile); // throws on any native failure
// after: fall back to NIO with explicit replace semantics
try {
  Storage.rename(tmpFile, destFile);
} catch (IOException e) {
  Files.move(tmpFile.toPath(), destFile.toPath(),
      StandardCopyOption.REPLACE_EXISTING,
      StandardCopyOption.ATOMIC_MOVE);
}

Defensive patterns

Strategy: fallback

Validate before calling

File parent = to.getParentFile();
if (parent != null && (!parent.isDirectory() || !Files.isWritable(parent.toPath()))) {
  throw new IOException("rename target dir unusable: " + parent);
}
if (to.exists() && !to.delete()) {
  throw new IOException("cannot replace existing destination: " + to);
}

Try / catch

try {
  Storage.rename(from, to);
} catch (IOException e) {
  // e carries the NativeIOException errno; only fall back when safe
  Files.move(from.toPath(), to.toPath(),
      StandardCopyOption.REPLACE_EXISTING, StandardCopyOption.ATOMIC_MOVE);
}

Prevention

Ensure the daemon user owns the storage directory and destination parent is writable
Remove stale destination files before rename-based operations
Keep src and dst on the same filesystem
Parse the errno in the message before retrying - permission errors never resolve by retrying

When it happens

Trigger: The daemon user lacks write permission on the storage directory (EACCES); the destination parent is missing; a cross-filesystem rename (EXDEV); on Windows, the destination exists and the move flag combination fails; SELinux denial on rename.

Common situations: Storage directory chowned to root or another user; directory on a read-only mount; leftover destination files from a crashed checkpoint; running daemons under a different user than the one that formatted storage.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a02c05fe782f7897. Report an issue: GitHub.