apache/hadoop · error · IOException

OutputFormat not set for FilterOutputFormat

Error message

OutputFormat not set for FilterOutputFormat

What it means

Thrown as IOException from FilterOutputFormat.getBaseOut (FilterOutputFormat.java:72). FilterOutputFormat is a pure delegating wrapper around a base OutputFormat; every method (getRecordWriter, checkOutputSpecs, getOutputCommitter) forwards to baseOut. If the wrapper was built with the no-arg constructor and baseOut was never assigned (it is protected, so subclasses may set it), the delegation cannot proceed and this IOException is thrown.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FilterOutputFormat.java:72

  throws IOException, InterruptedException {
    return getBaseOut().getRecordWriter(context);
  }

  @Override
  public void checkOutputSpecs(JobContext context) 
  throws IOException, InterruptedException {
    getBaseOut().checkOutputSpecs(context);
  }

  @Override
  public OutputCommitter getOutputCommitter(TaskAttemptContext context) 
  throws IOException, InterruptedException {
    return getBaseOut().getOutputCommitter(context);
  }

  private OutputFormat<K,V> getBaseOut() throws IOException {
    if (baseOut == null) {
      throw new IOException("OutputFormat not set for FilterOutputFormat");
    }
    return baseOut;
  }
  /**
   * <code>FilterRecordWriter</code> is a convenience wrapper
   * class that extends the {@link RecordWriter}.
   */

  public static class FilterRecordWriter<K,V> extends RecordWriter<K,V> {

    protected RecordWriter<K,V> rawWriter = null;

    public FilterRecordWriter() {
      rawWriter = null;
    }
    
    public FilterRecordWriter(RecordWriter<K,V> rwriter) {
      this.rawWriter = rwriter;

View on GitHub (pinned to 2add963021)

Solutions

1. Pass the delegate via the constructor: new FilterOutputFormat<>(new TextOutputFormat<>())
2. In subclasses, either call super(baseOut) or assign this.baseOut before the first delegating call
3. Add a unit test that calls checkOutputSpecs on a minimally configured job to catch the unset delegate at build time

Example fix

// before
class MetricsOutputFormat<K,V> extends FilterOutputFormat<K,V> {
  public MetricsOutputFormat() {} // baseOut stays null
}

// after
class MetricsOutputFormat<K,V> extends FilterOutputFormat<K,V> {
  public MetricsOutputFormat() {
    this.baseOut = new TextOutputFormat<>(); // or lazily from conf
  }
}

// constructor invariant: never allow an unwrapped FilterOutputFormat
public MyFilterOutputFormat(OutputFormat<K,V> base) {
  super(Objects.requireNonNull(base, "baseOut must be set"));
}

When it happens

Trigger: Subclassing FilterOutputFormat (directly, like LazyOutputFormat does) using the default constructor and never setting the protected baseOut field, then calling checkOutputSpecs/getRecordWriter/getOutputCommitter on it. Also instantiating FilterOutputFormat with new FilterOutputFormat<>() instead of new FilterOutputFormat<>(baseFormat).

Common situations: Custom wrapper OutputFormats (metrics, encryption, path-rewriting) whose author forgets to pass or assign the delegate; refactoring that drops the super(baseOut) call.

Related errors

• Record Writer not set for FilterRecordWriter
• SequenceFileAsBinaryOutputFormat doesn't support Record Comp
• Outputformat not set for FilterOutputFormat
• Record Writer not set for FilterRecordWriter
• Failure parsing JSON document: ${je.getMessage()}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d079ff95aa044e88. Report an issue: GitHub.