apache/hadoop · error · IOException

Job status not available

Error message

Job status not available 

What it means

Job.updateStatus() asks the attached Cluster's client (ResourceManager's MR client protocol, or a JobTracker) for the JobStatus of this job id. If the remote service returns null — it no longer knows the job — the method throws IOException('Job status not available ') (note the trailing space, useful for message matching). It surfaces through getStatus(), monitorAndPrintJob(), and getState().

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java:340

    
  /** Some methods need to update status immediately. So, refresh
   * immediately
   * @throws IOException
   */
  synchronized void updateStatus() throws IOException {
    try {
      this.status = ugi.doAs(new PrivilegedExceptionAction<JobStatus>() {
        @Override
        public JobStatus run() throws IOException, InterruptedException {
          return cluster.getClient().getJobStatus(getJobID());
        }
      });
    }
    catch (InterruptedException ie) {
      throw new IOException(ie);
    }
    if (this.status == null) {
      throw new IOException("Job status not available ");
    }
    this.statustime = System.currentTimeMillis();
  }
  
  public JobStatus getStatus() throws IOException, InterruptedException {
    ensureState(JobState.RUNNING);
    updateStatus();
    return status;
  }

  /**
   * Returns the current state of the Job.
   * 
   * @return JobStatus#State
   * @throws IOException
   * @throws InterruptedException
   */
  public JobStatus.State getJobState() 

View on GitHub (pinned to 2add963021)

Solutions

1. For finished jobs, query the job history layer instead of polling the live cluster (JobHistory API, jhist files, or the MR JobHistory UI)
2. Stop polling once job.isComplete() returns true — retired jobs will never come back from the RM
3. Confirm the client targets the cluster the job ran on (yarn.resourcemanager.address / mapreduce.jobtracker.address)
4. Right after submit(), tolerate the error once or twice with a short backoff — registration can lag

Example fix

// before
while (true) {
  JobStatus s = job.getStatus(); // throws 'Job status not available ' once the job is retired
  Thread.sleep(1000);
}

// after
while (!job.isComplete()) {
  JobStatus s = job.getStatus();
  Thread.sleep(1000);
}
// after completion use the history server for final details

// avoid polling a retired job: stop at completion
while (!job.isComplete()) {
  JobStatus s = job.getStatus(); // safe while the RM still tracks the job
  logProgress(s);
}

try {
  return job.getStatus();
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Job status not available")) {
    // job retired from the live cluster: fall back to the job history server
    return fetchStatusFromHistoryServer(job.getJobID());
  }
  throw e; // real connectivity problem — do not mask it
}

When it happens

Trigger: Calling job.getStatus() after the job finished and was retired from the RM's in-memory job table; using a Job id that belongs to a different or restarted cluster (RM restart loses job state); querying in the first instants after submit() before the MR ApplicationMaster registers the job.

Common situations: Long-lived monitoring threads or dashboards holding a Job object for days; RM failover/restart during a poll; pointing the client at the JobHistoryServer address where the active ResourceManager is expected; job ids carried across environments (dev vs prod configs).

Related errors

• "Mkdirs failed to create " + reduceIn.getParent().toString()
• "Couldn't rename " + mapOut
• "Couldn't rename " + mapOutIndex
• Not yet implemented.
• "Mkdirs failed to create " + workDir.toString()

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a9aed02c9adca926. Report an issue: GitHub.