{"record":{"id":"d84c377419c9a90a","repo":"apache/hadoop","slug":"couldn-t-rename-mapoutindex","errorCode":null,"errorMessage":"\"Couldn't rename \" + mapOutIndex","messagePattern":"\"Couldn't rename \" \\+ mapOutIndex","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java","lineNumber":585,"sourceCode":"    // move map output to reduce input\n    Path mapOut = subMapOutputFile.getOutputFile();\n    FileStatus mStatus = localFs.getFileStatus(mapOut);\n    Path reduceIn = subMapOutputFile.getInputFileForWrite(\n        TypeConverter.fromYarn(mapId).getTaskID(), mStatus.getLen());\n    Path mapOutIndex = subMapOutputFile.getOutputIndexFile();\n    Path reduceInIndex = new Path(reduceIn.toString() + \".index\");\n    if (LOG.isDebugEnabled()) {\n      LOG.debug(\"Renaming map output file for task attempt {} from original location {}\"\n              + \" to destination {}\", mapId, mapOut, reduceIn);\n    }\n    if (!localFs.mkdirs(reduceIn.getParent())) {\n      throw new IOException(\"Mkdirs failed to create \"\n          + reduceIn.getParent().toString());\n    }\n    if (!localFs.rename(mapOut, reduceIn))\n      throw new IOException(\"Couldn't rename \" + mapOut);\n    if (!localFs.rename(mapOutIndex, reduceInIndex))\n      throw new IOException(\"Couldn't rename \" + mapOutIndex);\n\n    return new RenamedMapOutputFile(reduceIn);\n  }\n\n  private static class RenamedMapOutputFile extends MapOutputFile {\n    private Path path;\n    \n    public RenamedMapOutputFile(Path path) {\n      this.path = path;\n    }\n    \n    @Override\n    public Path getOutputFile() throws IOException {\n      return path;\n    }\n\n    @Override\n    public Path getOutputFileForWrite(long size) throws IOException {","sourceCodeStart":567,"sourceCodeEnd":603,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/LocalContainerLauncher.java#L567-L603","documentation":"Companion to the map-output rename in LocalContainerLauncher: after successfully renaming the map output data file, the code renames the matching spill index file (mapOutIndex -> reduceIn + '.index'). If RawLocalFileSystem.rename returns false for that second file, the job throws IOException('Couldn't rename <mapOutIndex>') and the map attempt fails. Because the data file was already moved, the slot is now half-populated and a rerun needs cleanup.","triggerScenarios":"Stale '.index' destination file from an earlier attempt; index file removed by an aggressive cleanup thread mid-rename; ownership/permission differing between the .out and .out.index files; cross-volume allocation of the pair.","commonSituations":"Retried uber attempts after disk-full events leave partial reduce-input slots; antivirus or tmpwatch-style janitors deleting .index files under local dirs; mixed ownership under usercache.","solutions":["Inspect the exact mapOutIndex path from the message and its destination; delete the leftover reduceIn and reduceIn.index pair from the failed attempt before retrying","Fix ownership/permissions so both .out and .out.index share the same writable directory","Exclude MR local dirs from janitor/antivirus scans, or move mapreduce.cluster.local.dir out of cleaned temp paths","If it recurs, disable uber mode so tasks get isolated, freshly localized containers"],"exampleFix":"# cleanup of a half-moved slot before rerunning the job (paths from the error message)\n# before: failed attempt left reduce input pair inconsistent\nls -l /mrlocal/usercache/alice/appcache/application_123/output/attempt_*\n# after: remove the stale pair and resubmit\nrm -rf /mrlocal/usercache/alice/appcache/application_123/output/attempt_<failed>\nyarn jar myjob.jar -Dmapreduce.job.ubertask.enable=false ...","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"Catch IOException on the index-file rename and, because the data file already moved, remove BOTH reduceIn and reduceIn.index before the attempt is retried -- otherwise the next run fails on the stale pair.","preventionTips":["Treat a failed index rename as a partially migrated slot: clean both files","Keep janitor/cleanup daemons away from active MR local dirs","Watch for repeated half-renames as a symptom of disk or permission rot"],"tags":["mapreduce","yarn","uber-mode","rename","local-filesystem","partial-state"],"backgroundTag":"file-rename-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}