apache/hadoop · error · IllegalStateException

Job in state {} instead of {}

Error message

Job in state {} instead of {}

What it means

org.apache.hadoop.mapreduce.Job is a small state machine (DEFINE on construction, RUNNING after submit(), then finished). Most setters call ensureState(JobState.DEFINE) and status/control methods call ensureState(JobState.RUNNING). When a method is invoked from the wrong state, ensureState throws IllegalStateException naming the actual state and the required one (e.g. 'Job in state RUNNING instead of DEFINE').

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java:301

   * parameter.
   * 
   * @param cluster cluster
   * @param status job status
   * @param conf job configuration
   * @return the {@link Job} , with no connection to a cluster yet.
   * @throws IOException
   */
  @Private
  public static Job getInstance(Cluster cluster, JobStatus status, 
      Configuration conf) throws IOException {
    Job job = getInstance(status, conf);
    job.setCluster(cluster);
    return job;
  }

  private void ensureState(JobState state) throws IllegalStateException {
    if (state != this.state) {
      throw new IllegalStateException("Job in state "+ this.state + 
                                      " instead of " + state);
    }

    if (state == JobState.RUNNING && cluster == null) {
      throw new IllegalStateException
        ("Job in state " + this.state
         + ", but it isn't attached to any job tracker!");
    }
  }

  /**
   * Some methods rely on having a recent job status object.  Refresh
   * it, if necessary
   */
  synchronized void ensureFreshStatus() 
      throws IOException {
    if (System.currentTimeMillis() - statustime > MAX_JOBSTATUS_AGE) {
      updateStatus();

View on GitHub (pinned to 2add963021)

Solutions

1. Treat Job as single-use: build a new Job.getInstance(conf), do ALL configuration, then submit once
2. For a rerun with changes, write values into the Configuration first, then construct a fresh Job from it
3. Move getStatus/killTask/monitorAndPrintJob calls strictly after submit(), and gate polling with job.isComplete()
4. If you see 'RUNNING ... but it isn't attached to any job tracker', obtain the Job from a Cluster (Cluster.getJob(id)) so it has a client attached

Example fix

// before
job.submit();
if (runFailed) {
  job.setNumReduceTasks(1); // IllegalStateException: Job in state RUNNING instead of DEFINE
  job.submit();
}

// after
if (runFailed) {
  conf.setInt(MRJobConfig.NUM_REDUCES, 1);
  job = Job.getInstance(conf); // fresh DEFINE-state job
  job.submit();
}

// single-use Job wrapper that prevents wrong-state calls
final boolean[] submitted = {false};
Runnable safeSubmit = () -> {
  if (submitted[0]) throw new IllegalStateException("Job already submitted; create a new Job for a rerun");
  job.submit();
  submitted[0] = true;
};
// all setters must be called before safeSubmit.run()

try {
  job.setNumReduceTasks(2);
} catch (IllegalStateException e) {
  // message shape: 'Job in state RUNNING instead of DEFINE'
  if (e.getMessage().contains("instead of")) {
    throw new IllegalStateException("Configure the job before submit(), or build a new Job", e);
  }
  throw e;
}

When it happens

Trigger: Calling setNumReduceTasks/setMapperClass/setInputPath/setProfileTaskRange after job.submit() (state has moved to RUNNING); calling getStatus(), killTask(), or monitorAndPrintJob() on a freshly built Job that was never submitted (still DEFINE); a second check in ensureState also fires when a RUNNING job has no Cluster attached (job obtained via getInstance(status, conf) without a cluster).

Common situations: Retry logic that catches a failed run, tweaks the same Job object and resubmits it; wrappers that poll status before submit; code migrated from the old mapred JobConf API, where a JobConf could be freely reused across submissions.

Related errors

• ${name}: for this operation, the current service state must
• ${name} cannot enter state ${proposed} from state ${state}
• Invalid specification for distributed-cache artifacts of typ
• SequenceFileAsBinaryOutputFormat doesn't support Record Comp
• Invalid state of the job for cleanup. State found " + jobRun

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/03991c0a7158e52d. Report an issue: GitHub.