{"record":{"id":"368e8b5785079890","repo":"apache/iceberg","slug":"cannot-find-a-partition-spec-in-iceberg-table-s-t-368e8b","errorCode":null,"errorMessage":"Cannot find a partition spec in Iceberg table %s that matches the partition columns (%s) in input table","messagePattern":"Cannot find a partition spec in Iceberg table (.+?) that matches the partition columns \\((.+?)\\) in input table","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java","lineNumber":952,"sourceCode":"        partitionNames.stream()\n            .map(name -> name.toLowerCase(Locale.ROOT))\n            .collect(Collectors.toList());\n    for (PartitionSpec icebergSpec : icebergTable.specs().values()) {\n      boolean allIdentity =\n          icebergSpec.fields().stream().allMatch(field -> field.transform().isIdentity());\n      if (allIdentity) {\n        List<String> icebergPartNames =\n            icebergSpec.fields().stream()\n                .map(PartitionField::name)\n                .map(name -> name.toLowerCase(Locale.ROOT))\n                .collect(Collectors.toList());\n        if (icebergPartNames.equals(partitionNamesLower)) {\n          return icebergSpec;\n        }\n      }\n    }\n\n    throw new IllegalArgumentException(\n        String.format(\n            \"Cannot find a partition spec in Iceberg table %s that matches the partition\"\n                + \" columns (%s) in input table\",\n            icebergTable, partitionNames));\n  }\n\n  /**\n   * Returns the first partition spec in an IcebergTable that shares the same names and ordering as\n   * the partition columns in a given Spark Table. Throws an error if not found\n   */\n  private static PartitionSpec findCompatibleSpec(\n      Table icebergTable, SparkSession spark, String sparkTable) throws AnalysisException {\n    List<String> parts = Lists.newArrayList(Splitter.on('.').limit(2).split(sparkTable));\n    String db = parts.size() == 1 ? \"default\" : parts.get(0);\n    String table = parts.get(parts.size() == 1 ? 0 : 1);\n\n    List<String> sparkPartNames =\n        spark.catalog().listColumns(db, table).collectAsList().stream()","sourceCodeStart":934,"sourceCodeEnd":970,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java#L934-L970","documentation":"Thrown by SparkTableUtil's partition-spec matching helper when importing a Spark (Hive) table into Iceberg: none of the Iceberg table's partition specs has the same set of partition column names as the input table's partition columns (case-insensitively). Migration requires an exactly matching existing spec; Iceberg will not silently repartition the data.","triggerScenarios":"Running spark_table_util.importSparkTable / SparkTableUtil.partitionDF-by-partition filtering when the source table's partition columns do not match any spec in the target Iceberg table — e.g. source partitioned by (ds, hr) but Iceberg table is unpartitioned or partitioned by different columns/transforms.","commonSituations":"Migrating Hive tables where the Iceberg target was created unpartitioned; column name case or ordering differs; source uses date strings while Iceberg spec uses different column names; adding partitions to an already-imported unpartitioned table.","solutions":["Create (or recreate) the Iceberg table with PARTITIONED BY columns matching the source table's partition columns exactly.","If the source should be imported unpartitioned, drop the partitioning expectation and import without partition filters, or import partition-by-partition with a matching spec.","Align column names (case-insensitively) between the source table and the Iceberg spec before importing."],"exampleFix":"// before\nCREATE TABLE iceberg.db.t AS SELECT ...  -- unpartitioned\nCALL iceberg.system.import_spark_table(... partitioned by ds ...)\n// after\nCREATE TABLE iceberg.db.t USING iceberg PARTITIONED BY (ds)\nCALL iceberg.system.import_spark_table(...)","handlingStrategy":"validation","validationCode":"java.util.Set<String> src = new java.util.HashSet<>();\nfor (String p : partitionNames) src.add(p.toLowerCase(java.util.Locale.ROOT));\njava.util.Set<String> tgt = new java.util.HashSet<>();\nfor (Types.NestedField f : icebergSpec.partitionType().fields()) tgt.add(f.name().toLowerCase(java.util.Locale.ROOT));\nif (!src.equals(tgt)) throw new IllegalArgumentException(\"partition columns must match Iceberg spec: \" + src + \" vs \" + tgt);","typeGuard":null,"tryCatchPattern":"try { importSparkTable(...); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"partition spec\")) { /* recreate target with matching partitioning */ } else throw e; }","preventionTips":["Create the Iceberg target table PARTITIONED BY the same columns as the source Hive table before importing.","Compare partition column names case-insensitively across source and target.","Import unpartitioned tables without partition filters, or pre-plan the spec mapping."],"tags":["spark","migration","partitioning","table-import"],"backgroundTag":"schema-validation-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}