apache/beam · warning · IOException

Could not read because the thread got interrupted while…

Error message

Could not read because the thread got interrupted while reading the records with an exception: 

What it means

While the reader advances records with RecordReader.nextKeyValue(), an InterruptedException means the executing thread was interrupted (e.g., cancellation or runner shutdown). HadoopFormatIO converts it into this IOException, chaining the interrupt cause.

Solutions

1. Rerun the pipeline if interruption was due to cancellation or shutdown
2. Investigate why the runner interrupted the task (timeout, OOM kill, autoscaling)
3. Reduce per-record blocking in the custom RecordReader so interruption lands in a safe state

try {
  reader.advance();
} catch (IOException e) {
  if (e.getCause() instanceof InterruptedException) {
    Thread.currentThread().interrupt(); // restore interrupt flag
    return; // graceful shutdown
  }
  throw e;
}

When it happens

Trigger: Pipeline cancellation, job deadline exceeded, or runner shutdown while HadoopInputFormatReader.advance() is blocked inside nextKeyValue().

Common situations: Users canceling a running Beam pipeline; wall-clock timeouts in Dataflow/Flink causing task cancellation; long-blocking record readers being interrupted by the runner.

Related errors

• Unable to read data:
• Cannot create reader as source is not split yet.
• Error in computing the fractions consumed as…
• Cannot provide because does not implement the interface
• Cannot read format

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/66ddce67f8207ce2. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:941

        try {
          recordsReturned.set(0L);
          recordReader = inputFormatObj.createRecordReader(split.getSplit(), taskAttemptContext);
          if (recordReader != null) {
            recordReader.initialize(split.getSplit(), taskAttemptContext);
            progressValue.set(getProgress());
            if (recordReader.nextKeyValue()) {
              recordsReturned.incrementAndGet();
              doneReading = false;
              return true;
            }
          } else {
            throw new IOException(
                String.format(
                    "Null RecordReader object returned by %s", inputFormatObj.getClass()));
          }
          recordReader = null;
        } catch (InterruptedException e) {
          throw new IOException(
              "Could not read because the thread got interrupted while "
                  + "reading the records with an exception: ",
              e);
        }
        doneReading = true;
        return false;
      }

      @Override
      public boolean advance() throws IOException {
        try {
          progressValue.set(getProgress());
          if (recordReader.nextKeyValue()) {
            recordsReturned.incrementAndGet();
            return true;
          }
          doneReading = true;
        } catch (InterruptedException e) {

View on GitHub (pinned to 12126d8942)