apache/hadoop · error · RuntimeException

rename file failed

Error message

rename file failed

What it means

FileStore.completeUpload() concatenates all staged part files into a .tmp file next to the final object and then promotes it with tmpFile.renameTo(keyPath); if the rename returns false it throws 'rename file failed' — the assembled object could not replace/become the destination file even though every part was read and written successfully.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java:447

          throw new RuntimeException(
              String.format("part num mismatched: %d != %d", part.num(), partNums.get(i)));
        }

        File partFile = new File(uploadDir, String.valueOf(part.num()));
        checkPartFile(part, partFile);

        try (FileInputStream inputStream = new FileInputStream(partFile);
            FileChannel inputChannel = inputStream.getChannel()) {
          outputChannel.transferFrom(inputChannel, offset, partFile.length());
          offset += partFile.length();
        }
      }
    } catch (IOException e) {
      throw new RuntimeException(e);
    }

    if (!tmpFile.renameTo(keyPath.toFile())) {
      throw new RuntimeException("rename file failed");
    } else {
      try {
        FileUtils.deleteDirectory(uploadDir);
      } catch (IOException e) {
        LOG.warn("failed to clean upload directory.");
      }
    }

    return getFileChecksum(keyPath);
  }

  private byte[] getFileChecksum(Path keyPath) {
    return getFileMD5(keyPath);
  }

  private static byte[] getFileMD5(Path keyPath) {
    try {
      return DigestUtils.md5(Files.readAllBytes(keyPath));

View on GitHub (pinned to 2add963021)

Solutions

1. Ensure no reader holds the destination key open while the upload completes (coordinate writers/readers, or write to a new key then rename/copy at the object level)
2. Verify write permission on the destination directory and delete any stale read-only destination file before completing
3. Move the filestore root to a plain local filesystem (ext4/xfs/tmpfs), not NFS or network mounts
4. If the destination already exists and the platform refuses replace, delete it first (storage.delete(key)) then complete the upload again from a fresh uploadId

java.nio.file.Path dir = java.nio.file.Paths.get(root);
if (!java.nio.file.Files.isWritable(dir)) {
  throw new IOException("destination dir not writable: " + dir);
}
if (storage.objectStatus(key) != null) {
  storage.delete(key); // clear a possibly locked destination before completing
}

When it happens

Trigger: completeUpload reaching the final rename when the destination file is locked by a concurrent reader (typical on Windows), the object's directory is not writable, the destination exists with restrictive permissions, or the filesystem rejects the atomic replace (unusual mounts under the store root).

Common situations: Concurrent read of the same object key while a multipart write completes (Windows file locking); store root on NFS/FUSE mounts with unreliable rename semantics; leftover read-only destination file from a run under a different user; antivirus interfering on developer machines.

Related errors

• Failed to create MultipartUpload with key: %s
• cannot locate the upload id: %s
• failed to put file since rename fail.
• parts length mismatched: %d != %d
• part num mismatched: %d != %d

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1b5b85752c943676. Report an issue: GitHub.