apache/hadoop · error · IOException

Unknown Job {}

Error message

Unknown Job {}

What it means

MRClientService.verifyAndGetJob throws this IOException when a client of the AM's MRClientProtocol (getCounters, getTaskReports, killJob, ...) sends a JobId that appContext.getJob(jobID) does not know. A MapReduce AM hosts exactly one job for its application, so any other, stale, or malformed-but-parseable JobId is rejected before ACL checks run.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/client/MRClientService.java:207

  public int getHttpPort() {
    return webApp.port();
  }

  class MRClientProtocolHandler implements MRClientProtocol {

    private RecordFactory recordFactory = 
      RecordFactoryProvider.getRecordFactory(null);

    @Override
    public InetSocketAddress getConnectAddress() {
      return getBindAddress();
    }
    
    private Job verifyAndGetJob(JobId jobID, JobACL accessType,
        boolean exceptionThrow) throws IOException {
      Job job = appContext.getJob(jobID);
      if (job == null && exceptionThrow) {
        throw new IOException("Unknown Job " + jobID);
      }
      UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
      if (job != null && !job.checkAccess(ugi, accessType)) {
        throw new AccessControlException("User " + ugi.getShortUserName()
            + " cannot perform operation " + accessType.name() + " on "
            + jobID);
      }
      return job;
    }
 
    private Task verifyAndGetTask(TaskId taskID, 
        JobACL accessType) throws IOException {
      Task task =
          verifyAndGetJob(taskID.getJobId(), accessType, true).getTask(taskID);
      if (task == null) {
        throw new IOException("Unknown Task " + taskID);
      }
      return task;

View on GitHub (pinned to 2add963021)

Solutions

1. Re-fetch the Job handle from the running cluster (job = cluster.getJob(jobId)) and confirm the applicationId/attempt still matches before each AM RPC
2. Route status queries for finished or unknown jobs to the Job History Server (mapreduce.jobhistory.webapp.address) instead of the AM
3. If the error is immediate after submission, print the JobId actually used and compare against yarn application -list output for the app's tracking URL

// client-side: confirm the app (and its AM) is alive and owns the job before AM RPCs
ApplicationReport rpt = yarnClient.getApplicationReport(appId);
if (rpt.getYarnApplicationState().isFinalState()) {
  // finished: use JobHistoryServer, not the AM
}
Job job = cluster.getJob(JobID.forName(jobIdString));
if (job == null) { /* refresh or route to JHS instead of calling the AM */ }

When it happens

Trigger: Client uses a JobId from a previous AM attempt generation after the app was restarted; a tool polls the AM with a JobId taken from a different application; job state already transitioned past cleanup so the job was removed from appContext.

Common situations: Stale Cluster/Job handles in a long-running client after RM-triggered AM restart; job-status pollers mixing AM and JobHistoryServer endpoints; scripts that cache job IDs across resubmissions of the same workflow step.

Related errors

• Unknown Task {}
• Unknown TaskAttempt {}
• job, {}, is not found
• Not Found: {}
• JobId string : {} is not properly formed

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1ca7f5baa9cafa0a. Report an issue: GitHub.