What causes "unable to load configuration for job: {}" in apache/hadoop?

Check the ApplicationMaster log for the underlying IOException - it names the real path and cause

apache/hadoop · error · NotFoundException

unable to load configuration for job: {}

Error message

unable to load configuration for job: {}

What it means

getJobConf builds a ConfInfo from the job; internally the job loads its configuration XML (job.xml) from the filesystem it was submitted to, usually HDFS under the staging directory. An IOException during that read is converted to HTTP 404 'unable to load configuration for job: <jid>', hiding the real I/O cause.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java:335

    checkAccess(job, hsr);
    return new JobCounterInfo(this.appCtx, job);
  }

  @GET
  @Path("/jobs/{jobid}/conf")
  @Produces({ MediaType.APPLICATION_JSON + "; " + JettyUtils.UTF_8,
      MediaType.APPLICATION_XML + "; " + JettyUtils.UTF_8 })
  public ConfInfo getJobConf(@Context HttpServletRequest hsr,
      @PathParam("jobid") String jid) {

    init();
    Job job = getJobFromJobIdString(jid, appCtx);
    checkAccess(job, hsr);
    ConfInfo info;
    try {
      info = new ConfInfo(job);
    } catch (IOException e) {
      throw new NotFoundException("unable to load configuration for job: "
          + jid);
    }
    return info;
  }

  @GET
  @Path("/jobs/{jobid}/tasks")
  @Produces({ MediaType.APPLICATION_JSON + "; " + JettyUtils.UTF_8,
      MediaType.APPLICATION_XML + "; " + JettyUtils.UTF_8 })
  public TasksInfo getJobTasks(@Context HttpServletRequest hsr,
      @PathParam("jobid") String jid, @QueryParam("type") String type) {

    init();
    Job job = getJobFromJobIdString(jid, appCtx);
    checkAccess(job, hsr);
    TasksInfo allTasks = new TasksInfo();
    for (Task task : job.getTasks().values()) {
      TaskType ttype = null;

View on GitHub (pinned to 2add963021)

Solutions

Check the ApplicationMaster log for the underlying IOException - it names the real path and cause
Verify the file exists: hdfs dfs -ls <staging-dir>/<jobid>/job.xml; fix permissions if needed
If the job is finished, use the Job History Server REST endpoint /ws/v1/history/mapreduce/jobs/{jobid}/conf (port 19888)
Retry once HDFS recovers if the NameNode was the problem

Example fix

// before: finished job queried on the AM
GET http://am-host:8099/ws/v1/mapreduce/jobs/job_1401318629735_0001/conf
// 404 { "unable to load configuration for job: ..." }

// after: finished job served by the Job History Server
GET http://jhs-host:19888/ws/v1/history/mapreduce/jobs/job_1401318629735_0001/conf

Defensive patterns

Strategy: try-catch

Validate before calling

// Cheap pre-check: query job state first; only RUNNING jobs reliably have conf on the AM
JobInfo job = client.target(amBase).path("jobs/{jid}")
    .resolveTemplate("jid", jid).request(MediaType.APPLICATION_JSON)
    .get(JobInfo.class);
if (!"RUNNING".equals(job.getState())) {
  // use the Job History Server conf endpoint instead
}

Try / catch

try {
  return client.target(amBase).path("jobs/{jid}/conf")
      .resolveTemplate("jid", jid).request(MediaType.APPLICATION_JSON)
      .get(ConfInfo.class);
} catch (javax.ws.rs.NotFoundException nfe) {
  // fall back to the history server for finished jobs
  return client.target(jhsBase) // http://jhs:19888/ws/v1/history/mapreduce
      .path("jobs/{jid}/conf").resolveTemplate("jid", jid)
      .request(MediaType.APPLICATION_JSON).get(ConfInfo.class);
}

Prevention

Do not retain AM conf URLs beyond the job's lifetime - switch to the history server when a job finishes
Keep staging directories until the history server has captured job data
On secure clusters, confirm the AM's delegation tokens remain valid for the job duration

When it happens

Trigger: GET /ws/v1/mapreduce/jobs/{jobid}/conf while job.xml is missing (staging dir cleaned), unreadable (permissions), or the filesystem is down (NameNode unavailable); also on secure clusters when the AM's delegation tokens expired.

Common situations: Retired job still listed by the AM with its conf already reaped; automated HDFS cleanup of staging directories; HDFS outage during a long-running job; job viewed right after submission before conf visible.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/2c7cda109f475449. Report an issue: GitHub.