apache/beam · error · UnsupportedOperationException

Unsupported format: {format}

Error message

Unsupported format: {format}

What it means

AddFiles.fileMetrics selects a metrics extractor based on the detected FileFormat (PARQUET, ORC, AVRO). If the switch reaches default, the format is one the library cannot compute metrics for, so it throws UnsupportedOperationException with the format name. This guards against silently skipping metrics needed for partition inference.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/AddFiles.java:836

      FileFormat format,
      MetricsConfig config,
      NameMapping mapping,
      @Nullable ParquetMetadata preReadFooter) {
    switch (format) {
      case PARQUET:
        ParquetMetadata footer =
            checkStateNotNull(preReadFooter, "Parquet metrics require the pre-read footer");
        MessageType originalMessageType = footer.getFileMetaData().getSchema();
        if (!ParquetSchemaUtil.hasIds(originalMessageType)) {
          footer = getFooterWithTypeIds(originalMessageType, footer, mapping);
        }
        return ParquetUtil.footerMetrics(footer, Stream.empty(), config, mapping);
      case ORC:
        return OrcMetrics.fromInputFile(file, config, mapping);
      case AVRO:
        return new Metrics(Avro.rowCount(file), null, null, null, null);
      default:
        throw new UnsupportedOperationException("Unsupported format: " + format);
    }
  }

  /**
   * Some exceptions carry a null message (bare EOFException, NPE); the error-routing path must
   * never throw on one.
   */
  static String errorMessage(Throwable e) {
    return e.getMessage() != null ? e.getMessage() : e.toString();
  }

  /** Tries to infer other file formats. Defaults to Parquet. */
  public static FileFormat inferFormat(String path) {
    String lowerPath = path.toLowerCase();

    if (lowerPath.endsWith(".parquet") || lowerPath.endsWith(".pqt")) {
      return FileFormat.PARQUET;
    } else if (lowerPath.endsWith(".orc")) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Convert the files to Parquet/ORC/AVRO before adding them to the table
2. Upgrade the iceberg and beam-sdks-java-io-iceberg dependency versions so new formats are supported
3. Skip unsupported files explicitly and import them through a separate path
4. Log and filter files by extension during planning instead of letting the worker throw

Example fix

// before
ADD FILES FROM 'hdfs:///data/*.json'
// after
spark.read.json("hdfs:///data/").write.parquet("hdfs:///data_parquet/")
// then add files from data_parquet

Set<FileFormat> ok = Set.of(FileFormat.PARQUET, FileFormat.ORC, FileFormat.AVRO); files.removeIf(f -> !ok.contains(formatOf(f)));

null

try { metrics = fileMetrics(...); } catch (UnsupportedOperationException e) { log.warn("Skipping unsupported format", e); skip(file); }

When it happens

Trigger: Invoking AddFiles on a table whose data files are in a FileFormat outside PARQUET/ORC/AVRO (or a future/unknown enum constant).

Common situations: Importing files written in another format (e.g. MERGE/JSON) into an Iceberg table; custom FileFormat enum values from an Iceberg fork; format detection returning a value added in a newer Iceberg release than the Beam connector supports.

Related errors

• ORC file format not currently supported.
• Unknown File Format: {}
• Cannot read format: {}
• TODO: Add support for reading the timestamp from the encoded
• Un-globbable filesystem.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4adde436668c8292. Report an issue: GitHub.