What causes "Outputformat not set for FilterOutputFormat" in apache/hadoop?

In your FilterOutputFormat subclass, assign baseOut in the constructor or before first delegation (e.g. baseOut = new TextOutputFormat<>())

apache/hadoop · error · IOException

Outputformat not set for FilterOutputFormat

Error message

Outputformat not set for FilterOutputFormat

What it means

FilterOutputFormat is a wrapper (template) OutputFormat that delegates getRecordWriter() and checkOutputSpecs() to an inner OutputFormat held in its protected baseOut field. getBaseOut() throws this IOException when either method is invoked and baseOut was never assigned. It means the wrapper subclass was used without configuring the delegate it is supposed to wrap.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/FilterOutputFormat.java:65

   * @param out the underlying OutputFormat
   */
  public FilterOutputFormat (OutputFormat<K,V> out) {
    this.baseOut = out;
  }

  public RecordWriter<K, V> getRecordWriter(FileSystem ignored, JobConf job, 
      String name, Progressable progress) throws IOException {
    return getBaseOut().getRecordWriter(ignored, job, name, progress);
  }

  public void checkOutputSpecs(FileSystem ignored, JobConf job) 
  throws IOException {
    getBaseOut().checkOutputSpecs(ignored, job);
  }
  
  private OutputFormat<K,V> getBaseOut() throws IOException {
    if (baseOut == null) {
      throw new IOException("Outputformat not set for FilterOutputFormat");
    }
    return baseOut;
  }

  /**
   * <code>FilterRecordWriter</code> is a convenience wrapper
   * class that implements  {@link RecordWriter}.
   */

  public static class FilterRecordWriter<K,V> implements RecordWriter<K,V> {

    protected RecordWriter<K,V> rawWriter = null;

    public FilterRecordWriter() throws IOException {
      rawWriter = null;
    }

    public FilterRecordWriter(RecordWriter<K,V> rawWriter)  throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

In your FilterOutputFormat subclass, assign baseOut in the constructor or before first delegation (e.g. baseOut = new TextOutputFormat<>())
If you want output created only when records are written, use LazyOutputFormat.setOutputFormatClass(job, RealFormat.class) instead of hand-rolling a wrapper
If the subclass reads the delegate from a conf key, make sure that key is set before job submission
Fail fast in your subclass: throw a descriptive error in the constructor if the delegate cannot be resolved

Example fix

// before: subclass never sets the delegate
public class AuditOutputFormat<K,V> extends FilterOutputFormat<K,V> { }

// after: set baseOut before the framework calls getRecordWriter/checkOutputSpecs
public class AuditOutputFormat<K,V> extends FilterOutputFormat<K,V> {
  public AuditOutputFormat() {
    baseOut = new TextOutputFormat<K,V>();
  }
}

Defensive patterns

Strategy: validation

Validate before calling

// in a FilterOutputFormat subclass, fail loudly in the constructor
public MyFilterOutputFormat() {
  baseOut = new TextOutputFormat<K,V>();
  if (baseOut == null) throw new IllegalStateException("delegate output format required");
}

Try / catch

try {
  outputFormat.checkOutputSpecs(fs, conf);
} catch (IOException e) {
  if (e.getMessage().contains("not set for FilterOutputFormat")) {
    throw new IllegalStateException("FilterOutputFormat subclass never assigned baseOut", e);
  }
  throw e;
}

Prevention

Never use FilterOutputFormat subtypes without configuring their delegate — prefer composing a concrete OutputFormat
Prefer the shipped LazyOutputFormat.setOutputFormatClass over hand-written wrappers
Unit-test checkOutputSpecs() on a locally constructed JobConf before submitting

When it happens

Trigger: Writing a custom class that extends FilterOutputFormat but never assigns baseOut (or assigns it only lazily on a path not taken), then the framework calls checkOutputSpecs() or getRecordWriter() during job submission/task run. Instantiating the raw FilterOutputFormat type through configuration also hits this, since nothing sets baseOut.

Common situations: Copy-paste of LazyOutputFormat-like wrappers where the author forgets the baseOut = ... assignment; using a third-party OutputFormat that extends FilterOutputFormat but expects a configuration key that was never set (its lazy lookup returned null and it left baseOut null).

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/192e8e3473265423. Report an issue: GitHub.