What causes "Null RecordReader object returned by %s" in apache/beam?

Fix the InputFormat.getRecordReader() to throw a descriptive IOException instead of returning null

apache/beam · error · IOException

Null RecordReader object returned by

Error message

Null RecordReader object returned by %s

What it means

HadoopInputFormatReader's readNext, after exhausting the current RecordReader, asks the InputFormat for a RecordReader for the next split. If the InputFormat returns null instead of a RecordReader, this IOException is thrown, since the reader cannot proceed.

Solutions

Fix the InputFormat.getRecordReader() to throw a descriptive IOException instead of returning null
Ensure the record reader class set in the job configuration is valid and loadable
Verify the InputFormat supports all splits produced by its getSplits()

Example fix

// before
public RecordReader<K,V> getRecordReader(...) { if (bad) return null; ... }
// after
public RecordReader<K,V> getRecordReader(...) { if (bad) throw new IOException("No record reader for split"); ... }

Defensive patterns

Strategy: validation

Validate before calling

RecordReader<?,?> rr = inputFormat.getRecordReader(split, conf, reporter);
if (rr == null) throw new IOException("InputFormat returned null RecordReader for split " + split);

Try / catch

try {
  while (reader.advance()) { ... }
} catch (IOException e) {
  if (e.getMessage().startsWith("Null RecordReader object returned by")) {
    throw new IllegalStateException("Fix InputFormat.getRecordReader(): it must not return null", e);
  }
  throw e;
}

Prevention

Never return null from getRecordReader(); throw a descriptive IOException instead
Test the InputFormat with every split produced by getSplits()
Confirm the record reader class configured in the job conf exists

When it happens

Trigger: The configured InputFormat.getRecordReader() returns null for a split — typically a custom/buggy InputFormat or one that cannot handle the given SerializableSplit.

Common situations: Custom InputFormat whose getRecordReader() silently returns null on unsupported splits or on configuration problems; misconfigured record reader class in the job conf.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a2179b3721fe7f22. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:935

      public HadoopInputFormatBoundedSource<K, V> getCurrentSource() {
        return source;
      }

      @Override
      public boolean start() throws IOException {
        try {
          recordsReturned.set(0L);
          recordReader = inputFormatObj.createRecordReader(split.getSplit(), taskAttemptContext);
          if (recordReader != null) {
            recordReader.initialize(split.getSplit(), taskAttemptContext);
            progressValue.set(getProgress());
            if (recordReader.nextKeyValue()) {
              recordsReturned.incrementAndGet();
              doneReading = false;
              return true;
            }
          } else {
            throw new IOException(
                String.format(
                    "Null RecordReader object returned by %s", inputFormatObj.getClass()));
          }
          recordReader = null;
        } catch (InterruptedException e) {
          throw new IOException(
              "Could not read because the thread got interrupted while "
                  + "reading the records with an exception: ",
              e);
        }
        doneReading = true;
        return false;
      }

      @Override
      public boolean advance() throws IOException {
        try {
          progressValue.set(getProgress());

View on GitHub (pinned to 12126d8942)