{"record":{"id":"18a55e496fd3a3fa","repo":"apache/hadoop","slug":"mkdirs-failed-to-create-reducein-getparent","errorCode":null,"errorMessage":"\"Mkdirs failed to create \" + reduceIn.getParent().toString()","messagePattern":"\"Mkdirs failed to create \" \\+ reduceIn\\.getParent\\(\\)\\.toString\\(\\)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java","lineNumber":579,"sourceCode":"   * so there are no particular compatibility issues.)\n   */\n  @VisibleForTesting\n  protected static MapOutputFile renameMapOutputForReduce(JobConf conf,\n      TaskAttemptId mapId, MapOutputFile subMapOutputFile) throws IOException {\n    FileSystem localFs = FileSystem.getLocal(conf);\n    // move map output to reduce input\n    Path mapOut = subMapOutputFile.getOutputFile();\n    FileStatus mStatus = localFs.getFileStatus(mapOut);\n    Path reduceIn = subMapOutputFile.getInputFileForWrite(\n        TypeConverter.fromYarn(mapId).getTaskID(), mStatus.getLen());\n    Path mapOutIndex = subMapOutputFile.getOutputIndexFile();\n    Path reduceInIndex = new Path(reduceIn.toString() + \".index\");\n    if (LOG.isDebugEnabled()) {\n      LOG.debug(\"Renaming map output file for task attempt {} from original location {}\"\n              + \" to destination {}\", mapId, mapOut, reduceIn);\n    }\n    if (!localFs.mkdirs(reduceIn.getParent())) {\n      throw new IOException(\"Mkdirs failed to create \"\n          + reduceIn.getParent().toString());\n    }\n    if (!localFs.rename(mapOut, reduceIn))\n      throw new IOException(\"Couldn't rename \" + mapOut);\n    if (!localFs.rename(mapOutIndex, reduceInIndex))\n      throw new IOException(\"Couldn't rename \" + mapOutIndex);\n\n    return new RenamedMapOutputFile(reduceIn);\n  }\n\n  private static class RenamedMapOutputFile extends MapOutputFile {\n    private Path path;\n    \n    public RenamedMapOutputFile(Path path) {\n      this.path = path;\n    }\n    \n    @Override","sourceCodeStart":561,"sourceCodeEnd":597,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java#L561-L597","documentation":"In an uberized MR job (mapreduce.job.ubertask.enable=true), map tasks run inside the AM's JVM and LocalContainerLauncher moves their outputs into reduce-input slots on the local filesystem. Before renaming, it calls localFs.mkdirs(reduceIn.getParent()) and throws IOException('Mkdirs failed to create <dir>') when the parent of the reduce input path cannot be created -- typically a full, read-only, or permission-denied local directory.","triggerScenarios":"Uber job on a node where the MR local dirs (mapreduce.cluster.local.dir) are full; the attempt output path's parent exists as a FILE left by a crashed earlier attempt; directory owned by another user so mkdirs returns false.","commonSituations":"Small jobs deliberately uberized to save containers, then failing on nodes with tight local disk; stale appcache/attempt directories after repeated failed runs; shared dev nodes with mixed ownership under local dirs.","solutions":["On the node running the AM, check space and permissions of the mapreduce.cluster.local.dir volumes (df -h, ls -ld on the printed path)","Disable uber mode (mapreduce.job.ubertask.enable=false) so tasks run in normal NM containers with fresh localization","Remove stale attempt directories blocking the parent path, and correct ownership of usercache/<user> trees","Free disk or add local-dir capacity on that node"],"exampleFix":"<!-- mapred-site.xml: avoid AM-local task execution on disk-constrained nodes -->\n<!-- before -->\n<property><name>mapreduce.job.ubertask.enable</name><value>true</value></property>\n<!-- after -->\n<property><name>mapreduce.job.ubertask.enable</name><value>false</value></property>","handlingStrategy":"try-catch","validationCode":"# pre-flight on the AM node before submitting an uberized job\nVOL=$(grep -A1 'mapreduce.cluster.local.dir' mapred-site.xml | tail -1 | sed 's/.*<value>//;s|<.*||' | cut -d, -f1)\ndf -h \"$VOL\" || exit 2\ntest -w \"$VOL\" || { echo \"local dir not writable: $VOL\" >&2; exit 2; }","typeGuard":null,"tryCatchPattern":"Catch IOException around the task-execution block in uber-mode code and surface the failing path; treat mkdirs/rename failures as node-health problems (fail over to a non-uber run) rather than retrying on the same node.","preventionTips":["Monitor free space and writability of mapreduce.cluster.local.dir volumes on nodes that host AMs","Prefer mapreduce.job.ubertask.enable=false where local disks are constrained","Purge stale usercache/appcache trees after crash loops"],"tags":["mapreduce","yarn","uber-mode","local-filesystem","mkdir","disk-full"],"backgroundTag":"cannot-create-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}