apache/beam · error · UnsupportedOperationException

ORC file format not currently supported.

Error message

ORC file format not currently supported.

What it means

RecordWriter's constructor selects an Iceberg DataWriter based on the table's FileFormat. Only AVRPC/Parquet-style writers are implemented; ORC is explicitly rejected with UnsupportedOperationException because the Beam Iceberg connector does not ship an ORC writer.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/RecordWriter.java:120

                .withKeyMetadata(keyMetadata)
                .overwrite()
                .build();
        break;
      case PARQUET:
        Parquet.DataWriteBuilder parquetBuilder =
            Parquet.writeData(outputFile)
                .forTable(table)
                .createWriterFunc(GenericParquetWriter::create)
                .withPartition(partitionKey)
                .withKeyMetadata(keyMetadata)
                .overwrite();
        if (writeProperties != null && !writeProperties.isEmpty()) {
          parquetBuilder.setAll(writeProperties);
        }
        icebergDataWriter = parquetBuilder.build();
        break;
      case ORC:
        throw new UnsupportedOperationException("ORC file format not currently supported.");
      default:
        throw new RuntimeException("Unknown File Format: " + fileFormat);
    }
    activeIcebergWriters.inc();
    LOG.info(
        "Opened {} writer for table '{}', partition {}. Writing to path: {}",
        fileFormat,
        table.name(),
        partitionKey,
        absoluteFilename);
  }

  public void write(Record record) {

    icebergDataWriter.write(record);
  }

  public void close() throws IOException {

View on GitHub (pinned to 12126d8942)

Solutions

1. Change the table's write format to Parquet: ALTER TABLE ... SET TBLPROPERTIES ('write.format.default'='parquet') or update table properties programmatically before writing
2. Create/use a Parquet-format Iceberg table for the Beam pipeline
3. Implement/patch RecordWriter to use Iceberg's ORC DataWriter (ORC.from(conf)) if ORC is mandatory
4. If the format comes from a catalog default, override the property per-table in your catalog configuration

Example fix

// before
spark.sql("ALTER TABLE my_table SET TBLPROPERTIES ('write.format.default'='orc')");
// after
spark.sql("ALTER TABLE my_table SET TBLPROPERTIES ('write.format.default'='parquet')");

if (table.properties().getOrDefault("write.format.default", "parquet").equalsIgnoreCase("orc")) {
  throw new IllegalArgumentException("Beam Iceberg IO cannot write ORC tables; use parquet");
}

null

try {
  writeResult = icebergIO.writeTo(table).expand(input);
} catch (UnsupportedOperationException e) {
  if (e.getMessage().contains("ORC")) {
    LOG.error("Target table is ORC; repoint pipeline to a parquet table");
  } else { throw e; }
}

When it happens

Trigger: Writing to an Iceberg table whose format is ORC — i.e. table.properties() format-version/write.format.default is 'orc' or the FileFormat passed to RecordWriter is FileFormat.ORC — while using the Beam Iceberg IO write path.

Common situations: Pointing Beam's Iceberg sink at an existing ORC table created by Spark/Flink; a table default write format set to ORC in table properties; migrating pipelines between engines with different default formats.

Related errors

• Unsupported format: {format}
• Unknown File Format: {}
• TODO: Add support for reading the timestamp from the encoded
• Invalid secret type %s, currently supported types: %s
• Unsupported secret manager: '%s'. Currently supported option

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4a31cee5851e3da5. Report an issue: GitHub.