{"record":{"id":"4390199942249c8d","repo":"apache/hadoop","slug":"index-file-for-the-log-of-taskid-doesn-t-e","errorCode":null,"errorMessage":"Index file for the log of \" + taskid + \" doesn't exist.","messagePattern":"Index file for the log of \" \\+ taskid \\+ \" doesn't exist\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"warning","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java","lineNumber":128,"sourceCode":"  private static LogFileDetail getLogFileDetail(TaskAttemptID taskid, \n                                                LogName filter,\n                                                boolean isCleanup) \n  throws IOException {\n    File indexFile = getIndexFile(taskid, isCleanup);\n    BufferedReader fis = new BufferedReader(new InputStreamReader(\n      SecureIOUtils.openForRead(indexFile, obtainLogDirOwner(taskid), null),\n      StandardCharsets.UTF_8));\n    //the format of the index file is\n    //LOG_DIR: <the dir where the task logs are really stored>\n    //stdout:<start-offset in the stdout file> <length>\n    //stderr:<start-offset in the stderr file> <length>\n    //syslog:<start-offset in the syslog file> <length>\n    LogFileDetail l = new LogFileDetail();\n    String str = null;\n    try {\n      str = fis.readLine();\n      if (str == null) { // the file doesn't have anything\n        throw new IOException(\"Index file for the log of \" + taskid\n            + \" doesn't exist.\");\n      }\n      l.location = str.substring(str.indexOf(LogFileDetail.LOCATION)\n          + LogFileDetail.LOCATION.length());\n      // special cases are the debugout and profile.out files. They are\n      // guaranteed\n      // to be associated with each task attempt since jvm reuse is disabled\n      // when profiling/debugging is enabled\n      if (filter.equals(LogName.DEBUGOUT) || filter.equals(LogName.PROFILE)) {\n        l.length = new File(l.location, filter.toString()).length();\n        l.start = 0;\n        fis.close();\n        return l;\n      }\n      str = fis.readLine();\n      while (str != null) {\n        // look for the exact line containing the logname\n        if (str.contains(filter.toString())) {","sourceCodeStart":110,"sourceCodeEnd":146,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java#L110-L146","documentation":"TaskLog reads each attempt's log.index file (mapping stdout/stderr/syslog offsets) before serving task logs. If the first readLine() on that index returns null — the file exists but is empty — this IOException is thrown, reporting the taskid. It effectively means the attempt's log index was never written or was truncated, so its logs cannot be located.","triggerScenarios":"Serving logs for a task attempt whose JVM was killed before flushing the index (kill -9, OOM), a log dir partially removed by userlog retention/cleanup, or manual deletion of log.index while leaving the attempt dir behind.","commonSituations":"Viewing logs of long-finished attempts past mapreduce.task.userlog.retain.hours/rm setup; attempts killed by ulimits; reading logs programmatically via TaskLog/TaskLogReader for a task that never really started.","solutions":["Treat it as 'logs unavailable': catch the IOException and surface a friendly message instead of failing the caller.","Verify the attempt actually ran and check whether userlog retention (mapreduce.task.userlog.retain.* / yarn.nodemanager.log-aggregation settings) already removed the logs.","Inspect the attempt's log dir on the node: if log.index is 0 bytes but the log files exist, the attempt died before writing the index — nothing to read.","Increase log retention or enable log aggregation if users routinely need older task logs."],"exampleFix":"// before\nLogFileDetail d = TaskLog.readTaskLog(TaskLog.LogName.SYSLOG, taskid, LogFilter.read);\n\n// after\ntry {\n  LogFileDetail d = TaskLog.readTaskLog(TaskLog.LogName.SYSLOG, taskid, LogFilter.read);\n} catch (IOException e) {\n  // logs unavailable for this attempt; report and continue\n}","handlingStrategy":"try-catch","validationCode":"// before reading an attempt's logs, check its log.index is non-trivial\nFile attemptDir = TaskLog.getAttemptDir(taskid);\nFile idx = new File(attemptDir, TaskLog.LogName.INDEX.toString());\nif (!idx.exists() || idx.length() == 0) {\n  LOG.warn(\"No log index for \" + taskid + \"; logs unavailable\");\n  return Collections.emptyList();\n}","typeGuard":null,"tryCatchPattern":"try {\n  LogFileDetail detail = TaskLog.readTaskLog(LogName.SYSLOG, taskid, LogFilter.read);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"Index file for the log of\")) {\n    // treat as logs-unavailable; report and continue rather than failing the caller\n  }\n}","preventionTips":["Degrade gracefully when task logs are missing — retention may have removed them.","Enable log aggregation or raise retain hours if users need old task logs.","Check that attempts actually started before fetching their logs."],"tags":["hadoop","mapreduce","task-logs","missing-file","log-retention"],"backgroundTag":"missing-log-file","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}