apache/hadoop · error · RuntimeException

failed to put file since rename fail.

Error message

failed to put file since rename fail.

What it means

After FileStore.copyInputStreamToFile() successfully writes a staged .tmp.<uuid> file, it promotes it to the destination with File.renameTo(); if the rename returns false it throws 'failed to put file since rename fail.' The copy succeeded but the atomic swap did not, so the destination never receives the data. renameTo() returning false on the same directory usually means permission trouble, a file lock on the target, or platform-specific rename restrictions.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/FileStore.java:399

  private static void copyInputStreamToFile(InputStream in, File partFile, long contentLength) {
    File tmpFile = createTmpFile(partFile);
    try (FileOutputStream out = new FileOutputStream(tmpFile)) {
      long copiedBytes = IOUtils.copyLarge(in, out, 0, contentLength);

      if (copiedBytes < contentLength) {
        throw new IOException(
            String.format("Unexpect end of stream, expected length:%s, actual:%s", contentLength,
                tmpFile.length()));
      }
    } catch (IOException e) {
      CommonUtils.runQuietly(() -> FileUtils.delete(tmpFile));
      throw new RuntimeException(e);
    } finally {
      CommonUtils.runQuietly(in::close);
    }

    if (!tmpFile.renameTo(partFile)) {
      throw new RuntimeException("failed to put file since rename fail.");
    }
  }

  @Override
  public byte[] completeUpload(String key, String uploadId, List<Part> uploadParts) {
    Preconditions.checkArgument(uploadParts != null && uploadParts.size() > 0,
        "upload parts cannot be null or empty.");
    File uploadDir = uploadPath(key, uploadId).toFile();
    if (!uploadDir.exists()) {
      throw new RuntimeException("cannot locate the upload id: " + uploadId);
    }

    List<Integer> partNums = listPartNums(uploadDir);
    if (partNums.size() != uploadParts.size()) {
      throw new RuntimeException(String.format("parts length mismatched: %d != %d",
          partNums.size(), uploadParts.size()));
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Check directory permissions on the store root and staging tree: they must be writable by the process user (chmod, chown)
2. On Windows or when a target already exists, delete the destination file first and retry the operation (Linux silently replaces; Windows refuses)
3. Move the store root off exotic mounts (NFS/FUSE) onto local disk (tmpfs/ext4)
4. Remove stale .tmp.* and part files from crashed runs inside __STAGING__ so renames never collide with locked files

java.nio.file.Path dir = java.nio.file.Paths.get(root);
if (!java.nio.file.Files.isWritable(dir)) {
  throw new IOException("store root not writable, renames will fail: " + dir);
}
// avoid locked destinations before multipart work on Windows
if (storage.objectStatus(key) != null && isWindows()) {
  storage.delete(key);
}

When it happens

Trigger: put() or uploadPart() where renameTo(partFile) fails: destination file locked by a concurrent reader (Windows/antivirus), no write permission on the directory, a stale destination file with restrictive permissions, or the tmp/destination straddling a weird mount inside the store root.

Common situations: Running tests on Windows while another process (AV scanner, another test thread) holds the part file open; store root placed on a filesystem with permission quirks (NFS, some container mounts); leftover read-only part files from a previous run owned by another user.

Related errors

• Failed to create root dir. %s
• failed to create tmp file
• rename file failed
• File not found %s
• %s is not appendable because append non-existed object with

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e5bb5d922dcdacf5. Report an issue: GitHub.