What causes "Record Writer not set for FilterRecordWriter" in apache/hadoop?

Initialize the delegate via the constructor: super(actualWriter)

apache/hadoop · error · IOException

Record Writer not set for FilterRecordWriter

Error message

Record Writer not set for FilterRecordWriter

What it means

Thrown as IOException from FilterRecordWriter.getRawWriter (FilterOutputFormat.java:106). FilterRecordWriter delegates write() and close() to a raw RecordWriter; if it was created with the no-arg constructor and rawWriter was never set (the field is protected), the first write() or close() call hits this guard. Note that LazyOutputFormat's internal LazyRecordWriter overrides write/close and assigns rawWriter itself, so this error comes from custom subclasses or direct misuse of FilterRecordWriter.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FilterOutputFormat.java:106

    
    public FilterRecordWriter(RecordWriter<K,V> rwriter) {
      this.rawWriter = rwriter;
    }
    
    @Override
    public void write(K key, V value) throws IOException, InterruptedException {
      getRawWriter().write(key, value);
    }

    @Override
    public void close(TaskAttemptContext context) 
    throws IOException, InterruptedException {
      getRawWriter().close(context);
    }
    
    private RecordWriter<K,V> getRawWriter() throws IOException {
      if (rawWriter == null) {
        throw new IOException("Record Writer not set for FilterRecordWriter");
      }
      return rawWriter;
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

Initialize the delegate via the constructor: super(actualWriter)
Or assign the protected rawWriter field lazily before first use (the LazyRecordWriter pattern: create on first write)
Override write()/close() only when you also guarantee rawWriter is non-null by the time they run

Example fix

// before
class CountingWriter<K,V> extends FilterOutputFormat.FilterRecordWriter<K,V> {
  public CountingWriter() {} // rawWriter never set -> write() throws
}

// after
class CountingWriter<K,V> extends FilterOutputFormat.FilterRecordWriter<K,V> {
  public CountingWriter(RecordWriter<K,V> inner) {
    super(inner);
  }
}

Defensive patterns

Strategy: validation

Validate before calling

// enforce the delegate at construction
class MyWriter<K,V> extends FilterOutputFormat.FilterRecordWriter<K,V> {
  MyWriter(RecordWriter<K,V> inner) { super(Objects.requireNonNull(inner)); }
}

Prevention

Always call super(rawWriter) when subclassing FilterRecordWriter
If creating the writer lazily, assign rawWriter on first write() like LazyRecordWriter does
Avoid the no-arg constructor unless you override write/close entirely

When it happens

Trigger: Extending FilterRecordWriter with the default constructor, overriding neither the assignment of rawWriter nor write/close, then calling write(key, value) or close(context). Also new FilterRecordWriter<>() used directly as a RecordWriter without wrapping anything.

Common situations: Custom writer wrappers (counting, validating, transforming) where the author forgets super(rawWriter) in the constructor or to lazily create the writer in an overridden write().

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/03d490e42b2a285f. Report an issue: GitHub.