apache/iceberg · error · UnsupportedOperationException

Unknown partition format:

Error message

Unknown partition format: 

What it means

TableMigrationUtil.listPartition lists data files in a partition and computes metrics per supported format (parquet, avro, orc). If the partition's file format string matches none of these, an UnsupportedOperationException 'Unknown partition format: <format>' is thrown — the format is not supported for migration.

Solutions

1. Convert unsupported files to parquet/orc/avro before running the migration.
2. Pass the format in lowercase ('parquet', 'avro', 'orc') to listPartition.
3. Split the migration so partitions with unsupported formats are excluded or handled separately.
4. Verify the format argument against actual file extensions in the partition.

Example fix

// before
TableMigrationUtil.listPartition(list, partitionValues, "JSON", spec, conf, metricsSpec, mapping);

// after: convert data first, or use a supported format
TableMigrationUtil.listPartition(list, partitionValues, "parquet", spec, conf, metricsSpec, mapping);

Set<String> SUPPORTED = Set.of("parquet", "avro", "orc");
if (!SUPPORTED.contains(format.toLowerCase(Locale.ROOT))) {
  throw new IllegalArgumentException("Format not migratable: " + format);
}

boolean isMigratableFormat(String format) {
  return format != null
      && Set.of("parquet", "avro", "orc").contains(format.toLowerCase(Locale.ROOT));
}

try {
  TableMigrationUtil.listPartition(list, values, format, spec, conf, metricsSpec, mapping);
} catch (UnsupportedOperationException e) {
  if (e.getMessage().startsWith("Unknown partition format")) { convertOrSkipPartition(); }
}

When it happens

Trigger: Calling listPartition (used by add_files/migration procedures) with a partition whose format is not 'parquet', 'avro', or 'orc' (case-sensitive), e.g. 'json', 'csv', 'text', or uppercase 'PARQUET'.

Common situations: Migrating legacy Hive tables containing non-Iceberg-supported formats (JSON/CSV/Text); inconsistent file extensions or mixed formats in one partition; format string derived with wrong case from table properties.

Related errors

• Cannot read using unregistered internal data format
• Cannot write using unregistered internal data format
• Altering a view is not supported by catalog:
• Altering a view is not supported by catalog
• Altering a view is not supported by catalog

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5e556c919ff8eaec. Report an issue: GitHub.

Appendix: source

Thrown at data/src/main/java/org/apache/iceberg/data/TableMigrationUtil.java:202

            });
      } else if (format.contains("parquet")) {
        task.run(
            index -> {
              Metrics metrics =
                  getParquetMetrics(fileStatus.get(index).getPath(), conf, metricsSpec, mapping);
              datafiles[index] =
                  buildDataFile(fileStatus.get(index), partitionValues, spec, metrics, "parquet");
            });
      } else if (format.contains("orc")) {
        task.run(
            index -> {
              Metrics metrics =
                  getOrcMetrics(fileStatus.get(index).getPath(), conf, metricsSpec, mapping);
              datafiles[index] =
                  buildDataFile(fileStatus.get(index), partitionValues, spec, metrics, "orc");
            });
      } else {
        throw new UnsupportedOperationException("Unknown partition format: " + format);
      }
      return Arrays.asList(datafiles);
    } catch (IOException e) {
      throw new UncheckedIOException("Unable to list files in partition: " + partitionUri, e);
    } finally {
      if (service != null) {
        service.shutdown();
      }
    }
  }

  private static Metrics getAvroMetrics(Path path, Configuration conf) {
    try {
      InputFile file = HadoopInputFile.fromPath(path, conf);
      long rowCount = Avro.rowCount(file);
      return new Metrics(rowCount, null, null, null, null);
    } catch (UncheckedIOException e) {
      throw new RuntimeException("Unable to read Avro file: " + path, e);

View on GitHub (pinned to 86d9c8fc54)