apache/beam · error · RuntimeException

Unknown File Format: {}

Error message

Unknown File Format: {}

What it means

When RecordWriter's switch over FileFormat matches neither PARQUET/AVRO nor ORC, the default branch throws RuntimeException("Unknown File Format: ..."). This is a defensive internal check — FileFormat should be one of the known Iceberg enum values, so reaching default indicates a corrupted or synthetic format value.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/RecordWriter.java:122

                .build();
        break;
      case PARQUET:
        Parquet.DataWriteBuilder parquetBuilder =
            Parquet.writeData(outputFile)
                .forTable(table)
                .createWriterFunc(GenericParquetWriter::create)
                .withPartition(partitionKey)
                .withKeyMetadata(keyMetadata)
                .overwrite();
        if (writeProperties != null && !writeProperties.isEmpty()) {
          parquetBuilder.setAll(writeProperties);
        }
        icebergDataWriter = parquetBuilder.build();
        break;
      case ORC:
        throw new UnsupportedOperationException("ORC file format not currently supported.");
      default:
        throw new RuntimeException("Unknown File Format: " + fileFormat);
    }
    activeIcebergWriters.inc();
    LOG.info(
        "Opened {} writer for table '{}', partition {}. Writing to path: {}",
        fileFormat,
        table.name(),
        partitionKey,
        absoluteFilename);
  }

  public void write(Record record) {

    icebergDataWriter.write(record);
  }

  public void close() throws IOException {
    IOException closeError = null;
    try {

View on GitHub (pinned to 12126d8942)

Solutions

1. Log/inspect the table properties and ensure write.format.default is one of parquet, avro, orc (orc still unsupported by Beam, use parquet)
2. Null-check the FileFormat before constructing RecordWriter and default to FileFormat.PARQUET
3. Upgrade/align the Iceberg runtime version so format parsing matches the library expectations
4. Check for null fileFormat coming from TableProperties.WRITE_FORMAT_DEFAULT parsing and set a valid value

Example fix

// before
FileFormat format = FileFormat.valueOf(props.get("write.format.default")); // may be null
// after
FileFormat format = Optional.ofNullable(props.get("write.format.default"))
    .map(FileFormat::fromName).orElse(FileFormat.PARQUET);

FileFormat fmt = FileFormat.fromName(
    table.properties().getOrDefault(TableProperties.WRITE_FORMAT_DEFAULT, TableProperties.WRITE_FORMAT_DEFAULT_DEFAULT));
if (fmt == null) throw new IllegalArgumentException("Unrecognized write.format.default on table " + table.name());

null

try {
  RecordWriter writer = new RecordWriter(..., fileFormat, ...);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("Unknown File Format")) {
    LOG.error("Falling back to PARQUET writer");
    writer = new RecordWriter(..., FileFormat.PARQUET, ...);
  } else { throw e; }
}

When it happens

Trigger: fileFormat is null or an unrecognized value when RecordWriter is constructed — e.g. table.properties write.format.default parsed into a null FileFormat, a future/unknown format enum from a newer Iceberg table property, or a null passed through a custom write path.

Common situations: A table property like 'write.format.default' set to an invalid string that resolves to null FileFormat; mixing Iceberg library versions where new formats exist; a bug in calling code passing null into RecordWriter.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• Received null value for required field '{fieldName}'.
• Could not find a partition transform for '{}'.
• Could not find a partition term for '{}'.
• ORC file format not currently supported.
• Unrecognized value for stable unique names:

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a6de74f2857bf986. Report an issue: GitHub.