apache/iceberg · error · java.lang.IllegalArgumentException

Cannot find a partition spec in Iceberg table %s that matche

Error message

Cannot find a partition spec in Iceberg table %s that matches the partition columns (%s) in input table

What it means

When importing Spark (Hive/managed) partitioned tables into Iceberg, SparkTableUtil matches the input table's partition columns against one of the Iceberg table's partition specs. If no Iceberg spec's field names (lowercased) equal the input partition columns, import fails with this IllegalArgumentException. Iceberg only imports data into an existing spec that already covers those columns.

Source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java:1044

        partitionNames.stream()
            .map(name -> name.toLowerCase(Locale.ROOT))
            .collect(Collectors.toList());
    for (PartitionSpec icebergSpec : icebergTable.specs().values()) {
      boolean allIdentity =
          icebergSpec.fields().stream().allMatch(field -> field.transform().isIdentity());
      if (allIdentity) {
        List<String> icebergPartNames =
            icebergSpec.fields().stream()
                .map(PartitionField::name)
                .map(name -> name.toLowerCase(Locale.ROOT))
                .collect(Collectors.toList());
        if (icebergPartNames.equals(partitionNamesLower)) {
          return icebergSpec;
        }
      }
    }

    throw new IllegalArgumentException(
        String.format(
            "Cannot find a partition spec in Iceberg table %s that matches the partition"
                + " columns (%s) in input table",
            icebergTable, partitionNames));
  }

  /**
   * Returns the first partition spec in an IcebergTable that shares the same names and ordering as
   * the partition columns in a given Spark Table. Throws an error if not found
   */
  private static PartitionSpec findCompatibleSpec(
      Table icebergTable, SparkSession spark, String sparkTable) throws AnalysisException {
    List<String> parts = Lists.newArrayList(Splitter.on('.').limit(2).split(sparkTable));
    String db = parts.size() == 1 ? "default" : parts.get(0);
    String table = parts.get(parts.size() == 1 ? 0 : 1);

    List<String> sparkPartNames =
        spark.catalog().listColumns(db, table).collectAsList().stream()

View on GitHub (pinned to 86d9c8fc54)

Solutions

Create the Iceberg target table with a partition spec whose field names exactly match the source partition columns (case-insensitive)
ALTER the Iceberg table's partitioning (replacing the spec) so a spec matches the source columns
Import into an unpartitioned table only if the source is unpartitioned
Verify partition column names/size with DESCRIBE on both tables before importing

Example fix

// before
catalog.createTable(ident, schema, PartitionSpec.unpartitioned());
SparkTableUtil.importSparkTable(sparkSession, sourceTable, targetTable); // throws: no matching spec
// after
PartitionSpec spec = PartitionSpec.builderFor(schema).identity("part_date").build(); // matches source columns
catalog.createTable(ident, schema, spec);
SparkTableUtil.importSparkTable(sparkSession, sourceTable, targetTable);

Defensive patterns

Strategy: validation

Validate before calling

List<String> srcParts = sourcePartitionNames.stream().map(String::toLowerCase).sorted().toList();
boolean matched = targetSpecs.stream().anyMatch(s ->
    s.fields().stream().map(f -> f.name().toLowerCase()).sorted().toList().equals(srcParts));

Type guard

if (!matched) {
  throw new IllegalArgumentException("Target Iceberg specs do not match source partition columns");
}

Try / catch

try {
  SparkTableUtil.importSparkTable(spark, source, target);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("Cannot find a partition spec")) {
    // recreate target table with a matching partition spec, then retry
  }
}

Prevention

Create target Iceberg tables with specs mirroring source partition columns
Compare DESCRIBE output of source and target before importing
Watch for case and transform differences in partition fields
Version-check import scripts against table schemas

When it happens

Trigger: Calling SparkTableUtil.importSparkTable / snapshot partitioned Spark data where the source table's partition column names don't match (case-insensitively) any partition spec in the target Iceberg table.

Common situations: Source renamed or reordered partition columns vs the Iceberg spec; importing into an unpartitioned Iceberg table; case or transform mismatch (e.g. source partitioned by day, Iceberg spec by month) changing spec matching; copy-pasted import config from another table.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/379b0d1c32d1fb0f. Report an issue: GitHub.