apache/flink · error · IOException

Could not write Record.

Error message

Could not write Record.

What it means

Wraps an InterruptedException raised by recordWriter.write(key, value) inside HadoopOutputFormat.writeRecord(). Flink re-throws it as an IOException to satisfy the OutputFormat contract. It indicates the write was interrupted, usually because the task was cancelled or failed, not that the record itself is invalid.

Source

Thrown at flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/HadoopOutputFormat.java:49

 * @param <K> Key Type
 * @param <V> Value Type
 */
@Public
public class HadoopOutputFormat<K, V> extends HadoopOutputFormatBase<K, V, Tuple2<K, V>> {

    private static final long serialVersionUID = 1L;

    public HadoopOutputFormat(
            org.apache.hadoop.mapreduce.OutputFormat<K, V> mapreduceOutputFormat, Job job) {
        super(mapreduceOutputFormat, job);
    }

    @Override
    public void writeRecord(Tuple2<K, V> record) throws IOException {
        try {
            this.recordWriter.write(record.f0, record.f1);
        } catch (InterruptedException e) {
            throw new IOException("Could not write Record.", e);
        }
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Inspect the wrapped InterruptedException in the logs to identify cancellation or failover as the interrupt source.
2. Verify the output filesystem (HDFS/S3) is reachable and writable from the TaskManager.
3. If interruptions recur during heavy writes, check for backpressure and output-commit contention (FileOutputCommitter).
4. Treat as expected if the job was intentionally cancelled.

try {
    hadoopOutputFormat.writeRecord(record);
} catch (IOException e) {
    if (e.getCause() instanceof InterruptedException) {
        // write interrupted — usually task cancel/fail; do not silently drop the record
        LOG.warn("Output write interrupted; record may be lost", e);
        throw e;
    } else {
        throw e;
    }
}

When it happens

Trigger: Produced in HadoopOutputFormat.writeRecord() when this.recordWriter.write(record.f0, record.f1) throws InterruptedException — e.g. the task was cancelled/failing during a write, the sink's underlying I/O was interrupted, or a downstream commit/flush blocked and got interrupted.

Common situations: Job cancellation mid-write; task failover interrupting the writer; slow output filesystem (HDFS/S3) where a write is interrupted by a timeout; a custom RecordWriter that throws InterruptedException on recoverable conditions.

Related errors

• Could not create RecordWriter.
• Could not close RecordReader.
• Could not get KeyValue pair.
• Could not get Splits.
• Could not create RecordReader.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/e61c5e09035b4b7f. Report an issue: GitHub.