apache/iceberg · error · RuntimeIOException

Failed to write record

Error message

Failed to write record %s

What it means

ParquetWriteAdapter.add(D) writes a record via the underlying ParquetWriter and wraps any IOException in a RuntimeIOException with the message 'Failed to write record <datum>'. It converts checked I/O failures during record writing into runtime exceptions so file-writer pipelines don't declare IOException.

Solutions

1. Check filesystem capacity/permissions and the output stream's liveness; then retry the task
2. Inspect the cause (getCause()) to find the underlying IOException and fix it
3. Validate that the record matches the write schema before calling add

Example fix

// before: writing after closing the appender
appender.close();
appender.add(record);
// after: write all records, then close
appender.add(record);
appender.close();

Preconditions.checkState(!closed, "Appender already closed");
// verify record matches write schema before add
schema.asStruct().validate(record);

try {
  adapter.add(datum);
} catch (RuntimeIOException e) {
  LOG.error("Record write failed, datum={}, cause={}", datum, e.getCause());
  throw e; // task must fail; retry at the commit/task level
}

When it happens

Trigger: Calling add(datum) when the underlying parquet-mr writer throws IOException — e.g. disk full, underlying output stream closed, encryption failure, or a serialization error surfaced as IOException by parquet-mr.

Common situations: Out-of-disk or quota exhaustion on the target filesystem; writing to a closed/aborted task output stream; malformed records that parquet-mr's page writer rejects with an I/O-backed exception.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• could not read page in col " + desc
• could not read page in col
• could not read page " + valueCount + " in col " + desc
• Error reading mini block.
• Failed to create Parquet file

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/6e19b0bf976031f3. Report an issue: GitHub.

Appendix: source

Thrown at parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriteAdapter.java:56

 * @deprecated use {@link org.apache.iceberg.parquet.ParquetWriter}
 */
@Deprecated
public class ParquetWriteAdapter<D> implements FileAppender<D> {
  private ParquetWriter<D> writer;
  private final MetricsConfig metricsConfig;
  private ParquetMetadata footer;

  public ParquetWriteAdapter(ParquetWriter<D> writer, MetricsConfig metricsConfig) {
    this.writer = writer;
    this.metricsConfig = metricsConfig;
  }

  @Override
  public void add(D datum) {
    try {
      writer.write(datum);
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to write record %s", datum);
    }
  }

  @Override
  public Metrics metrics() {
    Preconditions.checkState(footer != null, "Cannot produce metrics until closed");
    // Note: Metrics reported by this method do not contain a full set of available metrics.
    // Specifically, it lacks metrics not included in Parquet file's footer (e.g. NaN count)
    MessageType messageType = footer.getFileMetaData().getSchema();
    Schema schema = ParquetSchemaUtil.convert(messageType);
    return ParquetMetrics.metrics(schema, messageType, metricsConfig, footer, Stream.empty());
  }

  @Override
  public long length() {
    return writer.getDataSize();
  }

View on GitHub (pinned to 86d9c8fc54)