apache/iceberg · error · UncheckedIOException

Failed to flush row group

Error message

Failed to flush row group

What it means

ParquetWriter.flushRowGroup() finishes the current row group (writing pages and the row-group footer) and starts a new one; any IOException from writeStore.close()/flush operations on the Parquet internal writer is wrapped as UncheckedIOException. Since flushRowGroup is invoked by checkSize, evaluateRowGroupSize, and close(), this error can surface during normal writing or at commit time and typically indicates a storage write failure or a corrupt/closed internal writer state.

Source

Thrown at parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java:248

          recordCount + Math.min(interval, props.getMaxRowCountForPageSizeCheck());
    }
  }

  private void flushRowGroup(boolean finished) {
    try {
      if (recordCount > 0) {
        ensureWriterInitialized();
        writer.startBlock(recordCount);
        writeStore.flush();
        pageStore.flushToFileWriter(writer);
        writer.endBlock();
        if (!finished) {
          writeStore.close();
          startRowGroup();
        }
      }
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to flush row group", e);
    }
  }

  private void startRowGroup() {
    Preconditions.checkState(!closed, "Writer is closed");

    this.nextCheckRecordCount =
        Math.min(
            Math.max(recordCount / 2, props.getMinRowCountForPageSizeCheck()),
            props.getMaxRowCountForPageSizeCheck());
    this.recordCount = 0;
    this.rowGroupUncompressedSize = 0;

    this.pageStore =
        new ColumnChunkPageWriteStore(
            compressor,
            parquetSchema,
            props.getAllocator(),

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Read e.getCause() for the underlying IOException (usually a storage write error or encoder failure)
2. Check target storage health: disk space, S3/GCS permissions, HDFS datanode availability
3. Reduce the configured write.row-group-size if huge buffered row groups stress memory/storage
4. Retry the write task — transient cloud I/O errors are common causes
5. If it happens at close(), verify the task wasn't already failed/cancelled and the stream closed

Example fix

// before: close() throws UncheckedIOException and aborts commit with no context
writer.close();

// after: surface the cause and clean up the partial file
try {
  writer.close();
} catch (UncheckedIOException e) {
  LOG.error("Flushing final row group failed: {}", e.getCause().getMessage(), e);
  io.deleteFile(file.location());
  throw e;
}

try { writer.close(); } catch (UncheckedIOException e) { io.deleteFile(file.location()); throw new RuntimeException("Row group flush failed: " + e.getCause().getMessage(), e); }

When it happens

Trigger: Row-group size threshold reached (row-group-size check), writer.close(), or metrics evaluation triggering flushRowGroup() while the underlying ParquetFileWriter/writeStore throws IOException during page serialization or footer write.

Common situations: Disk full or cloud-object-store write error when the row group is flushed; writing to a file already failed/closed; encoding failures on unusual data (e.g. dictionary page overflow with a broken encoder); metrics-based flush during a failing stream.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to read from input stream
• Failed to read bytes from stream
• Error reading mini block.
• Failed to read binary data
• could not read page in col " + desc

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/c650bf3c46373f69. Report an issue: GitHub.