{"record":{"id":"c650bf3c46373f69","repo":"apache/iceberg","slug":"failed-to-flush-row-group","errorCode":null,"errorMessage":"Failed to flush row group","messagePattern":"Failed to flush row group","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java","lineNumber":248,"sourceCode":"          recordCount + Math.min(interval, props.getMaxRowCountForPageSizeCheck());\n    }\n  }\n\n  private void flushRowGroup(boolean finished) {\n    try {\n      if (recordCount > 0) {\n        ensureWriterInitialized();\n        writer.startBlock(recordCount);\n        writeStore.flush();\n        pageStore.flushToFileWriter(writer);\n        writer.endBlock();\n        if (!finished) {\n          writeStore.close();\n          startRowGroup();\n        }\n      }\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Failed to flush row group\", e);\n    }\n  }\n\n  private void startRowGroup() {\n    Preconditions.checkState(!closed, \"Writer is closed\");\n\n    this.nextCheckRecordCount =\n        Math.min(\n            Math.max(recordCount / 2, props.getMinRowCountForPageSizeCheck()),\n            props.getMaxRowCountForPageSizeCheck());\n    this.recordCount = 0;\n    this.rowGroupUncompressedSize = 0;\n\n    this.pageStore =\n        new ColumnChunkPageWriteStore(\n            compressor,\n            parquetSchema,\n            props.getAllocator(),","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java#L230-L266","documentation":"ParquetWriter.flushRowGroup() finishes the current row group (writing pages and the row-group footer) and starts a new one; any IOException from writeStore.close()/flush operations on the Parquet internal writer is wrapped as UncheckedIOException. Since flushRowGroup is invoked by checkSize, evaluateRowGroupSize, and close(), this error can surface during normal writing or at commit time and typically indicates a storage write failure or a corrupt/closed internal writer state.","triggerScenarios":"Row-group size threshold reached (row-group-size check), writer.close(), or metrics evaluation triggering flushRowGroup() while the underlying ParquetFileWriter/writeStore throws IOException during page serialization or footer write.","commonSituations":"Disk full or cloud-object-store write error when the row group is flushed; writing to a file already failed/closed; encoding failures on unusual data (e.g. dictionary page overflow with a broken encoder); metrics-based flush during a failing stream.","solutions":["Read e.getCause() for the underlying IOException (usually a storage write error or encoder failure)","Check target storage health: disk space, S3/GCS permissions, HDFS datanode availability","Reduce the configured write.row-group-size if huge buffered row groups stress memory/storage","Retry the write task — transient cloud I/O errors are common causes","If it happens at close(), verify the task wasn't already failed/cancelled and the stream closed"],"exampleFix":"// before: close() throws UncheckedIOException and aborts commit with no context\nwriter.close();\n\n// after: surface the cause and clean up the partial file\ntry {\n  writer.close();\n} catch (UncheckedIOException e) {\n  LOG.error(\"Flushing final row group failed: {}\", e.getCause().getMessage(), e);\n  io.deleteFile(file.location());\n  throw e;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { writer.close(); } catch (UncheckedIOException e) { io.deleteFile(file.location()); throw new RuntimeException(\"Row group flush failed: \" + e.getCause().getMessage(), e); }","preventionTips":["Check disk/storage health before large write jobs; size row groups to fit memory and storage limits","Enable filesystem client retry policies for transient cloud I/O errors","Clean up partially written files on failure to avoid orphan data files","Keep writer.buffered-memory accounting sane so flushes are not deferred to a giant close()"],"tags":["parquet","io","row-group"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}