apache/iceberg · error · UncheckedIOException

Failed to start Parquet file writer

Error message

Failed to start Parquet file writer

What it means

ParquetWriter wraps the IOException thrown when Parquet's InternalParquetRecordWriter.start() fails to write the file header and begin the first row group. This happens after the underlying ParquetFileWriter has been created, so the failure is almost always caused by the underlying OutputFile/FileIO failing on write (disk full, permissions, credential expiry) or an internal Parquet WriterVersion/schema incompatibility. The IOException is rethrown as UncheckedIOException with the Parquet writer's own cause attached.

Source

Thrown at parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java:134

        this.writer =
            new ParquetFileWriter(
                ParquetIO.file(output, conf),
                parquetSchema,
                writeMode,
                targetRowGroupSize,
                0,
                columnIndexTruncateLength,
                ParquetProperties.DEFAULT_STATISTICS_TRUNCATE_LENGTH,
                ParquetProperties.DEFAULT_PAGE_WRITE_CHECKSUM_ENABLED,
                fileEncryptor);
      } catch (IOException e) {
        throw new UncheckedIOException("Failed to create Parquet file", e);
      }

      try {
        writer.start();
      } catch (IOException e) {
        throw new UncheckedIOException("Failed to start Parquet file writer", e);
      }
    }
  }

  @Override
  public void add(T value) {
    recordCount += 1;
    if (trackUncompressedSize) {
      writeTracked(value);
    } else {
      model.write(0, value);
    }
    writeStore.endRecord();
    checkSize();
  }

  private void writeTracked(T value) {
    long sizeBefore = writeStore.getBufferedSize();

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the chained cause (e.getCause()) — it carries the original IOException identifying the real storage-level failure
2. Verify the output location is writable and storage credentials (S3/HDFS/GCS tokens) are valid and not expired
3. Check available disk space or quota on the target filesystem
4. Confirm FileIO is correctly configured (e.g. correct warehouse path, no closed/invalid OutputFile)
5. Ensure the parquet-column dependency versions match across the classpath

Example fix

// before: writer created but storage credentials expired, UncheckedIOException propagates
ParquetWriter<T> writer = ParquetWriters.write(file, schema, ...);
writer.add(row); // throws "Failed to start Parquet file writer"

// after: validate writability upfront and handle the unchecked wrapper
try {
  writer.add(row);
} catch (UncheckedIOException e) {
  LOG.error("Cannot write to {}: {}", file.location(), e.getCause().getMessage());
  throw e;
}

try { writer.add(row); } catch (UncheckedIOException e) { LOG.error("Parquet write start failed: {}", e.getCause().getMessage()); throw e; }

When it happens

Trigger: Calling ParquetWriter.add(...) (or close) on a freshly created writer when ensureWriterInitialized lazily calls writer.start() and the underlying ParquetFileWriter.start() throws IOException — e.g. the output stream to the configured OutputFile cannot be written.

Common situations: HDFS/S3/local-disk write failures (disk full, permission denied, expired cloud credentials) surfacing when the first row group starts; a misconfigured FileIO or closed output stream; Parquet library version mismatch producing an internal failure during schema/materializer initialization.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to create Parquet output file for %s
• Failed to read from input stream
• Failed to read bytes from stream
• Error reading mini block.
• Failed to read binary data

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5ea0aa9a995810d6. Report an issue: GitHub.