{"record":{"id":"31e0bea97af4366c","repo":"apache/iceberg","slug":"cannot-write-using-unsupported-transforms-s-31e0be","errorCode":null,"errorMessage":"Cannot write using unsupported transforms: %s","messagePattern":"Cannot write using unsupported transforms: (.+?)","errorType":"exception","errorClass":"java.lang.UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/SparkUtil.java","lineNumber":100,"sourceCode":"\n  private SparkUtil() {}\n\n  /**\n   * Check whether the partition transforms in a spec can be used to write data.\n   *\n   * @param spec a PartitionSpec\n   * @throws UnsupportedOperationException if the spec contains unknown partition transforms\n   */\n  public static void validatePartitionTransforms(PartitionSpec spec) {\n    if (spec.fields().stream().anyMatch(field -> field.transform() instanceof UnknownTransform)) {\n      String unsupported =\n          spec.fields().stream()\n              .map(PartitionField::transform)\n              .filter(transform -> transform instanceof UnknownTransform)\n              .map(Transform::toString)\n              .collect(Collectors.joining(\", \"));\n\n      throw new UnsupportedOperationException(\n          String.format(\"Cannot write using unsupported transforms: %s\", unsupported));\n    }\n  }\n\n  /**\n   * A modified version of Spark's LookupCatalog.CatalogAndIdentifier.unapply Attempts to find the\n   * catalog and identifier a multipart identifier represents\n   *\n   * @param nameParts Multipart identifier representing a table\n   * @return The CatalogPlugin and Identifier for the table\n   */\n  public static <C, T> Pair<C, T> catalogAndIdentifier(\n      List<String> nameParts,\n      Function<String, C> catalogProvider,\n      BiFunction<String[], String, T> identiferProvider,\n      C currentCatalog,\n      String[] currentNamespace) {\n    Preconditions.checkArgument(","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/SparkUtil.java#L82-L118","documentation":"When planning a Spark write to an Iceberg table, the partition spec is validated to ensure every partition field uses a transform Spark can evaluate. If the spec contains an UnknownTransform (a transform from a newer/newer-unknown spec version or custom catalog), writes cannot route rows correctly and fail with UnsupportedOperationException listing the offending transforms.","triggerScenarios":"Writing to an Iceberg table whose partition spec was created by a newer Iceberg version (or another engine) using transforms this Spark writer does not recognize; the table's spec.fields() contains a PartitionField whose transform is UnknownTransform, checked via SparkUtil.validatePartitionTransforms during write planning.","commonSituations":"Reader/writer version skew: table created with a newer Iceberg release or with engine-specific partitioning (e.g. new bucket/day variants), then consumed by an older Spark runtime; cross-catalog table sharing where one side upgraded.","solutions":["Upgrade the Iceberg Spark runtime to a version that understands the table's transforms (align reader/writer versions)","Remove or rewrite the partition spec to use supported transforms (identity, bucket, truncate, years/months/days/hours) via ReplacePartitionSpec/update location","Avoid writing to that table from this Spark version; produce data through the engine that created the spec"],"exampleFix":"// before\n// built with newer iceberg-spark runtime, table has unknown transform\nspark.writeTo(\"catalog.db.table\").append();\n// after\n// upgrade dependency to matching version\n// build.gradle: implementation 'org.apache.iceberg:iceberg-spark-runtime-4.2_2.13:<matching-version>'","handlingStrategy":"validation","validationCode":"for (PartitionField field : table.spec().fields()) {\n  if (field.transform() instanceof UnknownTransform) {\n    throw new IllegalStateException(\"Unsupported partition transform: \" + field.transform());\n  }\n}","typeGuard":null,"tryCatchPattern":"try { df.writeTo(\"catalog.db.table\").append(); } catch (UnsupportedOperationException e) { if (e.getMessage().startsWith(\"Cannot write using unsupported transforms\")) { /* upgrade runtime or rewrite spec */ } else throw e; }","preventionTips":["Pin Iceberg runtime versions consistently across producers and consumers","Avoid writing to tables partitioned by transforms unknown to your runtime","Check spec compatibility after Iceberg upgrades"],"tags":["spark","partitioning","transform","version-skew"],"backgroundTag":"unsupported-operation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}