apache/iceberg · error · IllegalArgumentException

Cannot find a partition spec in Iceberg table %s that matche

Error message

Cannot find a partition spec in Iceberg table %s that matches the partition columns (%s) in input table

What it means

Thrown by SparkTableUtil's partition-spec matching helper when importing a Spark (Hive) table into Iceberg: none of the Iceberg table's partition specs has the same set of partition column names as the input table's partition columns (case-insensitively). Migration requires an exactly matching existing spec; Iceberg will not silently repartition the data.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java:952

        partitionNames.stream()
            .map(name -> name.toLowerCase(Locale.ROOT))
            .collect(Collectors.toList());
    for (PartitionSpec icebergSpec : icebergTable.specs().values()) {
      boolean allIdentity =
          icebergSpec.fields().stream().allMatch(field -> field.transform().isIdentity());
      if (allIdentity) {
        List<String> icebergPartNames =
            icebergSpec.fields().stream()
                .map(PartitionField::name)
                .map(name -> name.toLowerCase(Locale.ROOT))
                .collect(Collectors.toList());
        if (icebergPartNames.equals(partitionNamesLower)) {
          return icebergSpec;
        }
      }
    }

    throw new IllegalArgumentException(
        String.format(
            "Cannot find a partition spec in Iceberg table %s that matches the partition"
                + " columns (%s) in input table",
            icebergTable, partitionNames));
  }

  /**
   * Returns the first partition spec in an IcebergTable that shares the same names and ordering as
   * the partition columns in a given Spark Table. Throws an error if not found
   */
  private static PartitionSpec findCompatibleSpec(
      Table icebergTable, SparkSession spark, String sparkTable) throws AnalysisException {
    List<String> parts = Lists.newArrayList(Splitter.on('.').limit(2).split(sparkTable));
    String db = parts.size() == 1 ? "default" : parts.get(0);
    String table = parts.get(parts.size() == 1 ? 0 : 1);

    List<String> sparkPartNames =
        spark.catalog().listColumns(db, table).collectAsList().stream()

View on GitHub (pinned to 86d9c8fc54)

Solutions

Create (or recreate) the Iceberg table with PARTITIONED BY columns matching the source table's partition columns exactly.
If the source should be imported unpartitioned, drop the partitioning expectation and import without partition filters, or import partition-by-partition with a matching spec.
Align column names (case-insensitively) between the source table and the Iceberg spec before importing.

Example fix

// before
CREATE TABLE iceberg.db.t AS SELECT ...  -- unpartitioned
CALL iceberg.system.import_spark_table(... partitioned by ds ...)
// after
CREATE TABLE iceberg.db.t USING iceberg PARTITIONED BY (ds)
CALL iceberg.system.import_spark_table(...)

Defensive patterns

Strategy: validation

Validate before calling

java.util.Set<String> src = new java.util.HashSet<>();
for (String p : partitionNames) src.add(p.toLowerCase(java.util.Locale.ROOT));
java.util.Set<String> tgt = new java.util.HashSet<>();
for (Types.NestedField f : icebergSpec.partitionType().fields()) tgt.add(f.name().toLowerCase(java.util.Locale.ROOT));
if (!src.equals(tgt)) throw new IllegalArgumentException("partition columns must match Iceberg spec: " + src + " vs " + tgt);

Try / catch

try { importSparkTable(...); } catch (IllegalArgumentException e) { if (e.getMessage().contains("partition spec")) { /* recreate target with matching partitioning */ } else throw e; }

Prevention

Create the Iceberg target table PARTITIONED BY the same columns as the source Hive table before importing.
Compare partition column names case-insensitively across source and target.
Import unpartitioned tables without partition filters, or pre-plan the spec mapping.

When it happens

Trigger: Running spark_table_util.importSparkTable / SparkTableUtil.partitionDF-by-partition filtering when the source table's partition columns do not match any spec in the target Iceberg table — e.g. source partitioned by (ds, hr) but Iceberg table is unpartitioned or partitioned by different columns/transforms.

Common situations: Migrating Hive tables where the Iceberg target was created unpartitioned; column name case or ordering differs; source uses date strings while Iceberg spec uses different column names; adding partitions to an already-imported unpartitioned table.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/368e8b5785079890. Report an issue: GitHub.