apache/hadoop · error · InvalidJobConfException

Output directory not set.

Error message

Output directory not set.

What it means

Thrown as InvalidJobConfException (an IOException subclass) from FileOutputFormat.checkOutputSpecs (FileOutputFormat.java:156) when getOutputPath(job) returns null — i.e. mapreduce.output.fileoutputformat.outputdir is unset. checkOutputSpecs runs on the job client at submission, so the job fails before any mapper starts: FileOutputFormat refuses to run without a destination.

Solutions

  1. Call FileOutputFormat.setOutputPath(job, new Path("...")) before job.waitForCompletion()
  2. If the job truly needs no default file output, set an OutputFormat without this check: job.setOutputFormatClass(NullOutputFormat.class) (then configure MultipleOutputs separately)
  3. If output goes through MultipleOutputs only, still set a default output path or use LazyOutputFormat to avoid empty part files

Example fix

// before: FileOutputFormat-derived OutputFormat, no destination
job.setOutputFormatClass(TextOutputFormat.class);

// after: explicit output path
job.setOutputFormatClass(TextOutputFormat.class);
FileOutputFormat.setOutputPath(job, new Path(args[1]));
Defensive patterns

Strategy: validation

Validate before calling

// before submit
if (FileOutputFormat.getOutputPath(job) == null) {
  throw new IllegalArgumentException("Configure output path before submission");
}

Prevention

When it happens

Trigger: Submitting a job whose OutputFormat derives from FileOutputFormat while FileOutputFormat.setOutputPath(job, ...) was never called — e.g. only MultipleOutputs channels were configured, the setOutputPath line was dropped in a refactor, or the OutputFormat was switched from NullOutputFormat/DBOutputFormat to a FileOutputFormat without adding the path.

Common situations: Jobs that write exclusively through MultipleOutputs and forget the (still mandatory) default output path; pipelines where the output path is computed from an argument that came back null/empty; tutorial code adapted and the setOutputPath call removed.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/146acfb0df31ce8b. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.java:156

            conf.getClassByName(name).asSubclass(CompressionCodec.class);
      } catch (ClassNotFoundException e) {
        throw new IllegalArgumentException("Compression codec " + name + 
                                           " was not found.", e);
      }
    }
    return codecClass;
  }
  
  public abstract RecordWriter<K, V> 
     getRecordWriter(TaskAttemptContext job
                     ) throws IOException, InterruptedException;

  public void checkOutputSpecs(JobContext job
                               ) throws FileAlreadyExistsException, IOException{
    // Ensure that the output directory is set and not already there
    Path outDir = getOutputPath(job);
    if (outDir == null) {
      throw new InvalidJobConfException("Output directory not set.");
    }

    // get delegation token for outDir's file system
    TokenCache.obtainTokensForNamenodes(job.getCredentials(),
        new Path[] { outDir }, job.getConfiguration());

    if (outDir.getFileSystem(job.getConfiguration()).exists(outDir)) {
      throw new FileAlreadyExistsException("Output directory " + outDir + 
                                           " already exists");
    }
  }

  /**
   * Set the {@link Path} of the output directory for the map-reduce job.
   *
   * @param job The job to modify
   * @param outputDir the {@link Path} of the output directory for 
   * the map-reduce job.

View on GitHub (pinned to 2add963021)