{"record":{"id":"5ea0aa9a995810d6","repo":"apache/iceberg","slug":"failed-to-start-parquet-file-writer","errorCode":null,"errorMessage":"Failed to start Parquet file writer","messagePattern":"Failed to start Parquet file writer","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java","lineNumber":134,"sourceCode":"        this.writer =\n            new ParquetFileWriter(\n                ParquetIO.file(output, conf),\n                parquetSchema,\n                writeMode,\n                targetRowGroupSize,\n                0,\n                columnIndexTruncateLength,\n                ParquetProperties.DEFAULT_STATISTICS_TRUNCATE_LENGTH,\n                ParquetProperties.DEFAULT_PAGE_WRITE_CHECKSUM_ENABLED,\n                fileEncryptor);\n      } catch (IOException e) {\n        throw new UncheckedIOException(\"Failed to create Parquet file\", e);\n      }\n\n      try {\n        writer.start();\n      } catch (IOException e) {\n        throw new UncheckedIOException(\"Failed to start Parquet file writer\", e);\n      }\n    }\n  }\n\n  @Override\n  public void add(T value) {\n    recordCount += 1;\n    if (trackUncompressedSize) {\n      writeTracked(value);\n    } else {\n      model.write(0, value);\n    }\n    writeStore.endRecord();\n    checkSize();\n  }\n\n  private void writeTracked(T value) {\n    long sizeBefore = writeStore.getBufferedSize();","sourceCodeStart":116,"sourceCodeEnd":152,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/parquet/src/main/java/org/apache/iceberg/parquet/ParquetWriter.java#L116-L152","documentation":"ParquetWriter wraps the IOException thrown when Parquet's InternalParquetRecordWriter.start() fails to write the file header and begin the first row group. This happens after the underlying ParquetFileWriter has been created, so the failure is almost always caused by the underlying OutputFile/FileIO failing on write (disk full, permissions, credential expiry) or an internal Parquet WriterVersion/schema incompatibility. The IOException is rethrown as UncheckedIOException with the Parquet writer's own cause attached.","triggerScenarios":"Calling ParquetWriter.add(...) (or close) on a freshly created writer when ensureWriterInitialized lazily calls writer.start() and the underlying ParquetFileWriter.start() throws IOException — e.g. the output stream to the configured OutputFile cannot be written.","commonSituations":"HDFS/S3/local-disk write failures (disk full, permission denied, expired cloud credentials) surfacing when the first row group starts; a misconfigured FileIO or closed output stream; Parquet library version mismatch producing an internal failure during schema/materializer initialization.","solutions":["Inspect the chained cause (e.getCause()) — it carries the original IOException identifying the real storage-level failure","Verify the output location is writable and storage credentials (S3/HDFS/GCS tokens) are valid and not expired","Check available disk space or quota on the target filesystem","Confirm FileIO is correctly configured (e.g. correct warehouse path, no closed/invalid OutputFile)","Ensure the parquet-column dependency versions match across the classpath"],"exampleFix":"// before: writer created but storage credentials expired, UncheckedIOException propagates\nParquetWriter<T> writer = ParquetWriters.write(file, schema, ...);\nwriter.add(row); // throws \"Failed to start Parquet file writer\"\n\n// after: validate writability upfront and handle the unchecked wrapper\ntry {\n  writer.add(row);\n} catch (UncheckedIOException e) {\n  LOG.error(\"Cannot write to {}: {}\", file.location(), e.getCause().getMessage());\n  throw e;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { writer.add(row); } catch (UncheckedIOException e) { LOG.error(\"Parquet write start failed: {}\", e.getCause().getMessage()); throw e; }","preventionTips":["Validate storage credentials and output location writability before starting write tasks","Monitor disk space/quota on the target filesystem","Keep parquet/iceberg dependency versions aligned in the build","Always log e.getCause() so the real IOException is visible"],"tags":["parquet","io","file-write"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}