apache/hadoop · error · InvalidJobConfException

Output directory not set.

Error message

Output directory not set.

What it means

Thrown as InvalidJobConfException (an IOException subclass) from FileOutputFormat.checkOutputSpecs (FileOutputFormat.java:156) when getOutputPath(job) returns null — i.e. mapreduce.output.fileoutputformat.outputdir is unset. checkOutputSpecs runs on the job client at submission, so the job fails before any mapper starts: FileOutputFormat refuses to run without a destination.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.java:156

            conf.getClassByName(name).asSubclass(CompressionCodec.class);
      } catch (ClassNotFoundException e) {
        throw new IllegalArgumentException("Compression codec " + name + 
                                           " was not found.", e);
      }
    }
    return codecClass;
  }
  
  public abstract RecordWriter<K, V> 
     getRecordWriter(TaskAttemptContext job
                     ) throws IOException, InterruptedException;

  public void checkOutputSpecs(JobContext job
                               ) throws FileAlreadyExistsException, IOException{
    // Ensure that the output directory is set and not already there
    Path outDir = getOutputPath(job);
    if (outDir == null) {
      throw new InvalidJobConfException("Output directory not set.");
    }

    // get delegation token for outDir's file system
    TokenCache.obtainTokensForNamenodes(job.getCredentials(),
        new Path[] { outDir }, job.getConfiguration());

    if (outDir.getFileSystem(job.getConfiguration()).exists(outDir)) {
      throw new FileAlreadyExistsException("Output directory " + outDir + 
                                           " already exists");
    }
  }

  /**
   * Set the {@link Path} of the output directory for the map-reduce job.
   *
   * @param job The job to modify
   * @param outputDir the {@link Path} of the output directory for 
   * the map-reduce job.

View on GitHub (pinned to 2add963021)

Solutions

1. Call FileOutputFormat.setOutputPath(job, new Path("...")) before job.waitForCompletion()
2. If the job truly needs no default file output, set an OutputFormat without this check: job.setOutputFormatClass(NullOutputFormat.class) (then configure MultipleOutputs separately)
3. If output goes through MultipleOutputs only, still set a default output path or use LazyOutputFormat to avoid empty part files

Example fix

// before: FileOutputFormat-derived OutputFormat, no destination
job.setOutputFormatClass(TextOutputFormat.class);

// after: explicit output path
job.setOutputFormatClass(TextOutputFormat.class);
FileOutputFormat.setOutputPath(job, new Path(args[1]));

// before submit
if (FileOutputFormat.getOutputPath(job) == null) {
  throw new IllegalArgumentException("Configure output path before submission");
}

When it happens

Trigger: Submitting a job whose OutputFormat derives from FileOutputFormat while FileOutputFormat.setOutputPath(job, ...) was never called — e.g. only MultipleOutputs channels were configured, the setOutputPath line was dropped in a refactor, or the OutputFormat was switched from NullOutputFormat/DBOutputFormat to a FileOutputFormat without adding the path.

Common situations: Jobs that write exclusively through MultipleOutputs and forget the (still mandatory) default output path; pipelines where the output path is computed from an argument that came back null/empty; tutorial code adapted and the setOutputPath call removed.

Related errors

• Output directory {} already exists
• Invalid specification for distributed-cache artifacts of typ
• SequenceFileAsBinaryOutputFormat doesn't support Record Comp
• SequenceFileAsBinaryOutputFormat doesn't support Record Comp
• Failed to run job : {diagnostics}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/146acfb0df31ce8b. Report an issue: GitHub.