apache/hadoop · error · IOException

Not submitting job. Job directory {} already exists!! This i

Error message

Not submitting job. Job directory {} already exists!! This is unexpected.Please check what's there in that directory

What it means

JobResourceUploader.uploadResourcesInternal creates the per-job staging directory (<yarn.app.mapreduce.am.staging-dir>/<user>/.staging/job_<...>) and, being the first writer, first checks jtFs.exists(submitJobDir). A brand-new submission always receives a unique JobID, so an existing directory means the same job id is being submitted again; the code refuses with IOException('Not submitting job. Job directory ... already exists!!') to avoid clobbering the earlier run's artifacts.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java:165

    if (!(conf.getBoolean(Job.USED_GENERIC_PARSER, false))) {
      LOG.warn("Hadoop command-line option parsing not performed. "
          + "Implement the Tool interface and execute your application "
          + "with ToolRunner to remedy this.");
    }

    //
    // Figure out what fs the JobTracker is using. Copy the
    // job to it, under a temporary name. This allows DFS to work,
    // and under the local fs also provides UNIX-like object loading
    // semantics. (that is, if the job file is deleted right after
    // submission, we can still run the submission to completion)
    //

    // Create a number of filenames in the JobTracker's fs namespace
    LOG.debug("default FileSystem: " + jtFs.getUri());
    if (jtFs.exists(submitJobDir)) {
      throw new IOException("Not submitting job. Job directory " + submitJobDir
          + " already exists!! This is unexpected.Please check what's there in"
          + " that directory");
    }
    // Create the submission directory for the MapReduce job.
    submitJobDir = jtFs.makeQualified(submitJobDir);
    submitJobDir = new Path(submitJobDir.toUri().getPath());
    FsPermission mapredSysPerms =
        new FsPermission(JobSubmissionFiles.JOB_DIR_PERMISSION);
    mkdirs(jtFs, submitJobDir, mapredSysPerms);

    if (!conf.getBoolean(MRJobConfig.MR_AM_STAGING_DIR_ERASURECODING_ENABLED,
        MRJobConfig.DEFAULT_MR_AM_STAGING_ERASURECODING_ENABLED)) {
      disableErasureCodingForPath(submitJobDir);
    }

    // Get the resources that have been added via command line arguments in the
    // GenericOptionsParser (i.e. files, libjars, archives).
    Collection<String> files = conf.getStringCollection("tmpfiles");

View on GitHub (pinned to 2add963021)

Solutions

Never resubmit a submitted Job — allocate a new Job.getInstance(conf) for every attempt so a fresh JobID is minted
Delete the stale directory and retry: hdfs dfs -rm -r /tmp/hadoop-yarn/staging/<user>/.staging/job_XXXX then resubmit
Search your conf for a hardcoded mapreduce.job.id and remove it
If it recurs systematically, verify two processes are not submitting with the same id and that the staging dir is not shared across clusters

Example fix

// before
job.submit();
// later, after a transient failure:
job.submit(); // same JobID -> IOException: Job directory already exists!!

// after
job.submit();
// rerun path:
job = Job.getInstance(conf); // new JobID, new staging dir
job.submit();

Defensive patterns

Strategy: validation

Validate before calling

// reject accidental double-submit of one Job instance
if (submitted) {
  throw new IllegalStateException("This Job was already submitted; create a new Job.getInstance(conf)");
}
// optional: verify no stale dir for a pinned id before submitting
if (conf.get(MRJobConfig.JOB_ID) != null) {
  Path staging = new Path(conf.get("yarn.app.mapreduce.am.staging-dir"),
      Path.curPath... /* user */ + "/.staging/" + conf.get(MRJobConfig.JOB_ID));
  if (fs.exists(staging)) { /* refuse or clean */ }
}

Try / catch

try {
  job.submit();
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("already exists!!")) {
    throw new IOException("Staging dir for this JobID exists; delete it or submit a fresh Job", e);
  }
  throw e;
}

Prevention

One Job instance, one submit() call — encode this in your job-runner base class
Never copy mapreduce.job.id between submissions
Clean .staging leftovers only when no active submission is running

When it happens

Trigger: Calling job.submit() twice on the same Job object (the JobID was already allocated on the first call); a prior submission with the same id died after mkdir; a Configuration that pins mapreduce.job.id so two runs collide; two concurrent submitters reusing one fixed job id.

Common situations: Catch-and-resubmit retry loops that reuse the Job instance; staging leftovers under /tmp/hadoop-yarn/staging/<user>/.staging/ from a killed OOM'd submission; staging dirs shared between environments because yarn.app.mapreduce.am.staging-dir is misconfigured; scripts that copy a job.xml containing a literal mapreduce.job.id.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fb2060ce6741fa02. Report an issue: GitHub.