apache/hadoop · error · IOException

Unable to rename {src} to {dst}: couldn't create parent dire

Error message

Unable to rename {src} to {dst}: couldn't create parent directory

What it means

During map finalization, when exactly one spill file was produced, mergeParts() promotes that spill (and its index) to the final map output by renaming within the same volume. sameVolRename first ensures the destination's parent directory exists; if the parent is missing and mkdirs() fails - permissions, read-only mount, disk full, or a deleted attempt directory - this IOException is thrown.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java:2034

          rfs.delete(filename[i],true);
        }
      }
    }
    
    /**
     * Rename srcPath to dstPath on the same volume. This is the same
     * as RawLocalFileSystem's rename method, except that it will not
     * fall back to a copy, and it will create the target directory
     * if it doesn't exist.
     */
    private void sameVolRename(Path srcPath,
        Path dstPath) throws IOException {
      RawLocalFileSystem rfs = (RawLocalFileSystem)this.rfs;
      File src = rfs.pathToFile(srcPath);
      File dst = rfs.pathToFile(dstPath);
      if (!dst.getParentFile().exists()) {
        if (!dst.getParentFile().mkdirs()) {
          throw new IOException("Unable to rename " + src + " to "
              + dst + ": couldn't create parent directory");
        }
      }
      
      if (!src.renameTo(dst)) {
        throw new IOException("Unable to rename " + src + " to " + dst);
      }
    }

    /**
     * Increments numSpills local counter by taking into consideration
     * the max limit on spill files being generated by the job.
     * If limit is reached, this function throws an IOException
     */
    private void incrementNumSpills() throws IOException {
      ++numSpills;
      if(spillFilesCountLimit != SPILL_FILES_COUNT_UNBOUNDED_LIMIT_VALUE
          && numSpills > spillFilesCountLimit) {

View on GitHub (pinned to 2add963021)

Solutions

1. Check the parent path in the message: confirm the NM local dir it lives under exists, is writable, and has free space
2. Fix ownership/permissions of yarn.nodemanager.local-dirs on that host and restart the NodeManager if needed
3. Exclude or repair the volume if it has gone read-only
4. Rerun the job once the node is healthy

Example fix

# before: single local dir, full or unwritable by the NodeManager
yarn.nodemanager.local-dirs=/mnt/disk0/yarn-local

# after: multiple writable dirs with headroom, correct ownership
yarn.nodemanager.local-dirs=/mnt/disk0/yarn-local,/mnt/disk1/yarn-local
# chown yarn:yarn /mnt/disk*/yarn-local ; df -h shows free space

// NM health script excerpt: every local dir writable with a create/delete probe
for (String d : conf.getStrings("yarn.nodemanager.local-dirs")) {
  File probe = new File(d, ".probe-" + UUID.randomUUID());
  if (!probe.createNewFile()) throw new IOException("local dir not writable: " + d);
  probe.delete();
}

When it happens

Trigger: The numSpills == 1 fast path calls getOutputFileForWriteInVolume(filename[0]) and renames the spill into place; the target parent directory cannot be created because the NM local dir is full, unwritable, read-only after hardware trouble, or the attempt dir was removed underneath the task.

Common situations: Local-disk capacity or permission problems on the NodeManager; attempt directories pruned by aggressive cleanup while the task finishes; a volume remounted read-only; quota exhaustion on the spill partition.

Related errors

• Unable to rename {src} to {dst}
• "Mkdirs failed to create " + workDir.toString()
• Spill thread failed to initialize
• Buffer interrupted while waiting for the writer
• Interrupted while waiting for the writer

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7f08a0983b480059. Report an issue: GitHub.