What causes "Ouput format not set for LazyOutputFormat" in apache/hadoop?

Use LazyOutputFormat.setOutputFormatClass(job, TextOutputFormat.class) — it sets both the lazy wrapper and the underlying format key

apache/hadoop · error · IOException

Ouput format not set for LazyOutputFormat

Error message

Ouput format not set for LazyOutputFormat

What it means

LazyOutputFormat suppresses the creation of empty part files by wrapping a real OutputFormat that it reads from the job configuration key 'mapreduce.output.lazyoutputformat.outputformat'. getBaseOutputFormat() resolves that key and throws this IOException (note the original typo 'Ouput') when the configured class is null — i.e. the key was never set. It typically fires at checkOutputSpecs() time, which runs during job submission.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/LazyOutputFormat.java:75

    return new LazyRecordWriter<K, V>(job, baseOut, name, progress);
  }

  @Override
  public void checkOutputSpecs(FileSystem ignored, JobConf job) 
  throws IOException {
    if (baseOut == null) {
      getBaseOutputFormat(job);
    }
    super.checkOutputSpecs(ignored, job);
  }

  @SuppressWarnings("unchecked")
  private void getBaseOutputFormat(JobConf job) throws IOException {
    baseOut = ReflectionUtils.newInstance(
        job.getClass("mapreduce.output.lazyoutputformat.outputformat", null, OutputFormat.class), 
        job); 
    if (baseOut == null) {
      throw new IOException("Ouput format not set for LazyOutputFormat");
    }
  }
  
  /**
   * <code>LazyRecordWriter</code> is a convenience 
   * class that works with LazyOutputFormat.
   */

  private static class LazyRecordWriter<K,V> extends FilterRecordWriter<K,V> {

    final OutputFormat of;
    final String name;
    final Progressable progress;
    final JobConf job;

    public LazyRecordWriter(JobConf job, OutputFormat of, String name,
        Progressable progress)  throws IOException {
      this.of = of;

View on GitHub (pinned to 2add963021)

Solutions

Use LazyOutputFormat.setOutputFormatClass(job, TextOutputFormat.class) — it sets both the lazy wrapper and the underlying format key
If configuring by hand, set the key explicitly: job.setClass("mapreduce.output.lazyoutputformat.outputformat", TextOutputFormat.class, OutputFormat.class)
Verify in the submitted job.xml that mapreduce.output.lazyoutputformat.outputformat is present before launching

Example fix

// before: sets only the wrapper, underlying key missing -> IOException
job.setOutputFormatClass(LazyOutputFormat.class);

// after: sets LazyOutputFormat AND mapreduce.output.lazyoutputformat.outputformat
LazyOutputFormat.setOutputFormatClass(job, TextOutputFormat.class);

Defensive patterns

Strategy: validation

Validate before calling

// assert the lazy-output config before submitting
Class<?> base = conf.getClass("mapreduce.output.lazyoutputformat.outputformat", null, OutputFormat.class);
if (LazyOutputFormat.class.equals(conf.getOutputFormatClass()) && base == null) {
  throw new IllegalStateException("call LazyOutputFormat.setOutputFormatClass(conf, RealFormat.class)");
}

Prevention

Always configure lazy output through LazyOutputFormat.setOutputFormatClass(job, RealFormat.class) — never job.setOutputFormatClass(LazyOutputFormat.class)
After building the JobConf, grep it (conf.get on the key) as a cheap smoke test
Watch for this after refactors: renaming the real OutputFormat class usually leaves the key pointing at the old name

When it happens

Trigger: Calling job.setOutputFormatClass(LazyOutputFormat.class) manually instead of LazyOutputFormat.setOutputFormatClass(job, RealFormat.class). The static helper is the only code that sets 'mapreduce.output.lazyoutputformat.outputformat'; setting the format class directly leaves the key empty, so job.getClass(..., null, ...) returns null and the IOException is thrown.

Common situations: Migrating old jobs to use lazy output (often combined with MultipleOutputs to avoid empty part files); copying example code that sets the output format class directly; the key being lost when a JobConf is reconstructed from XML that never contained it.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a87ef47f71ed3fff. Report an issue: GitHub.