apache/iceberg · error · UncheckedIOException

Failed to create new data writer

Error message

Failed to create new data writer

What it means

RegistryBasedFileWriterFactory.newDataWriter builds a DataWriter via a writer registry; an IOException from opening/creating the output data file is wrapped in UncheckedIOException with this message. It means the factory could not produce a writer for the data file (open failure, unknown format, registry misconfiguration).

Source

Thrown at data/src/main/java/org/apache/iceberg/data/RegistryBasedFileWriterFactory.java:123

        table != null ? MetricsConfig.forTable(table) : MetricsConfig.getDefault();

    try {
      FileWriterBuilder<DataWriter<T>, S> builder =
          FormatModelRegistry.dataWriteBuilder(dataFileFormat, inputType, file);
      return builder
          .schema(dataSchema)
          .engineSchema(inputSchema())
          .setAll(properties)
          .setAll(writerProperties)
          .metricsConfig(metricsConfig)
          .spec(spec)
          .partition(partition)
          .keyMetadata(keyMetadata)
          .sortOrder(dataSortOrder)
          .overwrite()
          .build();
    } catch (IOException e) {
      throw new UncheckedIOException("Failed to create new data writer", e);
    }
  }

  @Override
  public EqualityDeleteWriter<T> newEqualityDeleteWriter(
      EncryptedOutputFile file, PartitionSpec spec, StructLike partition) {
    Preconditions.checkArgument(equalityDeleteRowSchema != null, "Invalid delete schema: null");

    EncryptionKeyMetadata keyMetadata = file.keyMetadata();
    Map<String, String> properties = table != null ? table.properties() : ImmutableMap.of();
    MetricsConfig metricsConfig =
        table != null ? MetricsConfig.forTable(table) : MetricsConfig.getDefault();

    try {
      FileWriterBuilder<EqualityDeleteWriter<T>, S> builder =
          FormatModelRegistry.equalityDeleteWriteBuilder(deleteFileFormat, inputType, file);
      return builder
          .setAll(properties)

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the chained IOException cause for the actual open failure (permissions, missing bucket, auth).
2. Verify the writer registry contains a factory for the table's file format (parquet/avro/orc).
3. Check output file location and FileIO credentials/permissions.
4. Fix encryption/keys config if encrypted output files are used.

Example fix

// before: registry lacks an avro writer
// after: register the writer before use
factoryRegistry.addFactory(Avro.DataWriter.class, new AvroDataWriterFactory());

// pre-check output writability before creating writers
try (OutputStream ignored = io.newOutputFile(location).create()) { }

try {
  DataWriter<T> writer = factory.newDataWriter(file, spec, partition, keyMetadata);
} catch (UncheckedIOException e) {
  log.error("Data writer creation failed", e.getCause());
  throw e;
}

When it happens

Trigger: Calling newDataWriter(file, spec, partition, keyMetadata) where opening the EncryptedOutputFile fails (bad path/bucket, credentials, encryption key error) or no writer is registered for the requested file format.

Common situations: Misconfigured write.output format or custom writer registry missing mappings; storage credentials expired; output location not writable; encryption keys misconfigured (encryption.key-id) without KMS access.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to create new equality delete writer
• Failed to create new position delete writer
• Failed to close current writer
• Failed to close encryption manager
• Failed to write json to file: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/9c598ab889d4d05a. Report an issue: GitHub.