apache/iceberg · error · IllegalArgumentException

Cannot find a partition spec in Iceberg table %s that matche

Error message

Cannot find a partition spec in Iceberg table %s that matches the partition columns (%s) in input table

What it means

SparkTableUtil.importSparkTable / partition-spec matching requires the Iceberg table's partition spec to cover exactly the same partition columns as the Spark (Hive) source table. When no Iceberg spec's partition-field names (lowercased) equal the input table's partition column set, this IllegalArgumentException is thrown rather than silently mis-importing partitioned data.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java:1044

        partitionNames.stream()
            .map(name -> name.toLowerCase(Locale.ROOT))
            .collect(Collectors.toList());
    for (PartitionSpec icebergSpec : icebergTable.specs().values()) {
      boolean allIdentity =
          icebergSpec.fields().stream().allMatch(field -> field.transform().isIdentity());
      if (allIdentity) {
        List<String> icebergPartNames =
            icebergSpec.fields().stream()
                .map(PartitionField::name)
                .map(name -> name.toLowerCase(Locale.ROOT))
                .collect(Collectors.toList());
        if (icebergPartNames.equals(partitionNamesLower)) {
          return icebergSpec;
        }
      }
    }

    throw new IllegalArgumentException(
        String.format(
            "Cannot find a partition spec in Iceberg table %s that matches the partition"
                + " columns (%s) in input table",
            icebergTable, partitionNames));
  }

  /**
   * Returns the first partition spec in an IcebergTable that shares the same names and ordering as
   * the partition columns in a given Spark Table. Throws an error if not found
   */
  private static PartitionSpec findCompatibleSpec(
      Table icebergTable, SparkSession spark, String sparkTable) throws AnalysisException {
    List<String> parts = Lists.newArrayList(Splitter.on('.').limit(2).split(sparkTable));
    String db = parts.size() == 1 ? "default" : parts.get(0);
    String table = parts.get(parts.size() == 1 ? 0 : 1);

    List<String> sparkPartNames =
        spark.catalog().listColumns(db, table).collectAsList().stream()

View on GitHub (pinned to 86d9c8fc54)

Solutions

Recreate the Iceberg table with a partition spec whose partition columns exactly match the source table's partition columns
Drop/rename partitions in the source or alter the target spec so the column name sets match exactly (names are compared lowercased)
If partitioning differences are intentional, import the data manually (e.g. via Spark writes) instead of SparkTableUtil's partition-spec matching path

Example fix

// before: Iceberg table partitioned by day(ts) but Hive table partitioned by column 'dt'
String.create(...)
// after: create Iceberg table partitioned by identity(dt)
Table table = catalog.createTable(ident, schema, PartitionSpec.builderFor(schema).identity("dt").build());

Defensive patterns

Strategy: validation

Validate before calling

Set<String> src = sourcePartitionCols.stream().map(c -> c.toLowerCase(Locale.ROOT)).collect(Collectors.toSet());
Set<String> target = table.spec().partitionType().fields().stream().map(f -> f.name().toLowerCase(Locale.ROOT)).collect(Collectors.toSet());
if (!src.equals(target)) throw new IllegalStateException("Partition columns differ: " + src + " vs " + target);

Try / catch

try { SparkTableUtil.importSparkTable(sparkSession, sourceTable, table, stagingDir); } catch (IllegalArgumentException e) { if (e.getMessage().contains("Cannot find a partition spec")) { /* recreate target with matching spec */ } throw e; }

Prevention

Create the Iceberg target with identity partitions on exactly the source table's partition columns
Compare partition column names (case-insensitively) before importing
Avoid transform-based specs on the target if you plan to import via SparkTableUtil from a Hive table

When it happens

Trigger: Calling SparkTableUtil.importSparkTable / getPartitions with a source Spark table whose partition columns do not exactly match any partition spec of the target Iceberg table (extra, missing, reordered, or differently named columns).

Common situations: Migrating a Hive table to Iceberg when the target Iceberg table was created with different partition columns (e.g. date vs day transform, different column name, unpartitioned target); typos in partition column names; case differences combined with renamed columns.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/0a070b4a8008a5c9. Report an issue: GitHub.