{"record":{"id":"4ab72b0e6a3d2ab7","repo":"apache/hadoop","slug":"cannot-get-log-path-for-a-in-progress-job","errorCode":null,"errorMessage":"Cannot get log path for a in-progress job","messagePattern":"Cannot get log path for a in-progress job","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java","lineNumber":534,"sourceCode":"            taReport.getContainerId().toString(),\n            taReport.getContainerId().getApplicationAttemptId()\n                .getApplicationId().toString(),\n            NodeId.newInstance(taReport.getNodeManagerHost(),\n                taReport.getNodeManagerPort()).toString(), report.getUser());\n      } else {\n        if (report.getAMInfos() == null || report.getAMInfos().size() == 0) {\n          throw new IOException(\"Unable to get log information for job: \"\n              + oldJobID);\n        }\n        AMInfo amInfo = report.getAMInfos().get(report.getAMInfos().size() - 1);\n        return new LogParams(\n            amInfo.getContainerId().toString(),\n            amInfo.getAppAttemptId().getApplicationId().toString(),\n            NodeId.newInstance(amInfo.getNodeManagerHost(),\n                amInfo.getNodeManagerPort()).toString(), report.getUser());\n      }\n    } else {\n      throw new IOException(\"Cannot get log path for a in-progress job\");\n    }\n  }\n\n  public void close() throws IOException {\n    if (rm != null) {\n      rm.close();\n    }\n\n    if (historyServerProxy != null) {\n      RPC.stopProxy(historyServerProxy);\n    }\n\n    if (realProxy != null) {\n      RPC.stopProxy(realProxy);\n      realProxy = null;\n    }\n  }\n}","sourceCodeStart":516,"sourceCodeEnd":552,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java#L516-L552","documentation":"Thrown by ClientServiceDelegate.getLogFilePath when the job's reported state is not one of SUCCEEDED/FAILED/KILLED/ERROR — i.e. the job is still running or pending. Log file parameters are only computed from a finalized report (container ids and NM locations are stable then); for in-progress jobs the API refuses with IOException by design.","triggerScenarios":"Calling Cluster.getLogFileParams/JobClient.getLogFileParams while the job is RUNNING, NEW, or ACCEPTED — before JobMonitor completion (e.g. calling from a thread that did not waitForCompletion).","commonSituations":"Calling getLogFileParams right after submitJob instead of after job completion; Polling loop that races the job's transition to a terminal state and queries logs on the last RUNNING poll; Reusing sample code that assumed a synchronous job run","solutions":["Wait for terminal state first: job.waitForCompletion() or poll JobStatus until isJobComplete(), then call getLogFileParams","Use `yarn logs -am <appId>` or `yarn logs -applicationId` to tail in-progress job logs instead","Guard the call: only invoke when EnumSet(SUCCEEDED,FAILED,KILLED,ERROR).contains(state)"],"exampleFix":"// before\nRunningJob r = jc.submitJob(conf);\nLogParams lp = client.getLogFileParams(jobId); // throws: still running\n\n// after\nJob job = jc.submitJob(conf);\njob.waitForCompletion();   // reach terminal state first\nLogParams lp = client.getLogFileParams(jobId);","handlingStrategy":"validation","validationCode":"JobStatus st = cluster.getJobStatus(jobId);\nboolean terminal = EnumSet.of(JobStatus.State.SUCCEEDED, JobStatus.State.FAILED,\n    JobStatus.State.KILLED).contains(st.getState()); // plus RUNTIME_FAILURE analogues\nif (!terminal) { /* wait or poll; do not call getLogFileParams yet */ }","typeGuard":"boolean isTerminal(JobStatus.State s) {\n  return s == JobStatus.State.SUCCEEDED || s == JobStatus.State.FAILED\n      || s == JobStatus.State.KILLED;\n}","tryCatchPattern":"try { lp = cluster.getLogFileParams(jobId); }\ncatch (IOException e) {\n  if (e.getMessage().contains(\"in-progress job\")) {\n    job.waitForCompletion(); lp = cluster.getLogFileParams(jobId);\n  } else throw e;\n}","preventionTips":["Always waitForCompletion() (or poll to terminal state) before requesting log file params","Use `yarn logs -am`/`yarn logs -applicationId` to tail running-job logs instead","In pollers, re-check state immediately before the call to avoid the RUNNING race"],"tags":["hadoop","mapreduce","yarn","logs","job-state","invalid-state"],"backgroundTag":"invalid-job-state","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}