What causes "Could not rename ${tmp} to ${file}" in apache/hadoop?

Ensure only one JHS process writes to the store.

apache/hadoop · error · IOException

Could not rename ${tmp} to ${file}

Error message

Could not rename ${tmp} to ${file}

What it means

createNewFile writes token/key data to a temp file (TMP_FILE_PREFIX) beside the final name and renames it into place so readers never observe partial files. If FileSystem.rename returns false, it throws IOException 'Could not rename' and deletes the temp file in the catch block, so no partial record survives but the store operation failed. Root causes are the same class of filesystem problems as the updateToken rename failure.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryServerFileSystemStateStoreService.java:249

      if (!status.isDirectory()) {
        throw new FileAlreadyExistsException("Unexpected file in store: "
            + dir);
      }
      if (!status.getPermission().equals(DIR_PERMISSIONS)) {
        fs.setPermission(dir, DIR_PERMISSIONS);
      }
    } catch (FileNotFoundException e) {
      fs.mkdirs(dir, DIR_PERMISSIONS);
    }
  }

  private void createNewFile(Path file, byte[] data)
      throws IOException {
    Path tmp = new Path(file.getParent(), TMP_FILE_PREFIX + file.getName());
    writeFile(tmp, data);
    try {
      if (!fs.rename(tmp, file)) {
        throw new IOException("Could not rename " + tmp + " to " + file);
      }
    } catch (IOException e) {
      fs.delete(tmp, false);
      throw e;
    }
  }

  private void writeFile(Path file, byte[] data) throws IOException {
    final int WRITE_BUFFER_SIZE = 4096;
    FSDataOutputStream out = fs.create(file, FILE_PERMISSIONS, true,
        WRITE_BUFFER_SIZE, fs.getDefaultReplication(file),
        fs.getDefaultBlockSize(file), null);
    try {
      try {
        out.write(data);
        out.close();
        out = null;
      } finally {

View on GitHub (pinned to 2add963021)

Solutions

Ensure only one JHS process writes to the store.
Check parent directory permissions for the JHS user on the path named in the message.
Verify filesystem health and free space, then restart JHS to retry the store operations.
Remove any partially-written leftovers before restart.

Defensive patterns

Strategy: try-catch

Try / catch

try {
  // store operation that creates token/key files
} catch (IOException e) {
  if (e.getMessage().startsWith("Could not rename")) {
    // temp file is auto-deleted; fix filesystem perms/health then retry via restart
  }
}

Prevention

Guarantee a single writer to the store.
Audit directory permissions for the JHS user after any security change.
Watch disk space on the store volume.

When it happens

Trigger: Target file created concurrently by a second writer; insufficient permission on the parent directory for rename; HDFS transient failure; parent directory removed between writeFile and rename.

Common situations: Two JHS instances pointed at one store; ACL or permission tightening after deployment; store directory on a failing or full disk.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/929565d05fbe9476. Report an issue: GitHub.