{"record":{"id":"27fe3c7d2ca54b7a","repo":"apache/iceberg","slug":"cannot-write-using-unsupported-transforms-s-27fe3c","errorCode":null,"errorMessage":"Cannot write using unsupported transforms: %s","messagePattern":"Cannot write using unsupported transforms: (.+?)","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/SparkUtil.java","lineNumber":100,"sourceCode":"\n  private SparkUtil() {}\n\n  /**\n   * Check whether the partition transforms in a spec can be used to write data.\n   *\n   * @param spec a PartitionSpec\n   * @throws UnsupportedOperationException if the spec contains unknown partition transforms\n   */\n  public static void validatePartitionTransforms(PartitionSpec spec) {\n    if (spec.fields().stream().anyMatch(field -> field.transform() instanceof UnknownTransform)) {\n      String unsupported =\n          spec.fields().stream()\n              .map(PartitionField::transform)\n              .filter(transform -> transform instanceof UnknownTransform)\n              .map(Transform::toString)\n              .collect(Collectors.joining(\", \"));\n\n      throw new UnsupportedOperationException(\n          String.format(\"Cannot write using unsupported transforms: %s\", unsupported));\n    }\n  }\n\n  /**\n   * A modified version of Spark's LookupCatalog.CatalogAndIdentifier.unapply Attempts to find the\n   * catalog and identifier a multipart identifier represents\n   *\n   * @param nameParts Multipart identifier representing a table\n   * @return The CatalogPlugin and Identifier for the table\n   */\n  public static <C, T> Pair<C, T> catalogAndIdentifier(\n      List<String> nameParts,\n      Function<String, C> catalogProvider,\n      BiFunction<String[], String, T> identiferProvider,\n      C currentCatalog,\n      String[] currentNamespace) {\n    Preconditions.checkArgument(","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/SparkUtil.java#L82-L118","documentation":"Iceberg writes require every partition field's transform to be understood by the writer. If a target table's partition spec contains an UnknownTransform (a transform the client cannot interpret — typically from a newer spec version or another writer), SparkUtil.validatePartitionTransforms refuses to write, listing the offending transforms.","triggerScenarios":"Running a Spark write (e.g. via df.writeTo(...).append()) against an Iceberg table whose partition spec includes fields whose transform resolves to UnknownTransform.","commonSituations":"Writing to a table created/upgraded by a newer Iceberg version or another engine that supports transforms this client does not; tables created via REST catalog with unrecognized transforms in the spec.","solutions":["Upgrade the Iceberg runtime (iceberg-spark) to a version that supports the transforms used in the table's spec","Rewrite the table with a partition spec using supported transforms (identity, year/month/day/hour, bucket, truncate)","Repartition the target table to drop the unknown-transform fields before writing"],"exampleFix":"// before: table partitioned by transform 'foo(col)' unknown to this client\ndf.writeTo(\"db.tbl\").append(); // throws\n// after: recreate table with supported spec\nPartitionSpec spec = PartitionSpec.builderFor(schema).day(\"ts\").build();","handlingStrategy":"validation","validationCode":"boolean hasUnknown = table.spec().fields().stream().anyMatch(f -> f.transform() instanceof UnknownTransform);\nif (hasUnknown) throw new IllegalStateException(\"Target table spec contains transforms unsupported by this client; upgrade Iceberg or repartition\");","typeGuard":null,"tryCatchPattern":"try { df.writeTo(\"db.tbl\").append(); } catch (UnsupportedOperationException e) { if (e.getMessage().contains(\"unsupported transforms\")) { /* upgrade runtime or repartition table */ } throw e; }","preventionTips":["Keep all writers on compatible Iceberg versions","Check spec transforms before pointing a Spark job at a foreign-created table","Prefer widely supported transforms (identity, bucket, truncate, temporal) when designing specs"],"tags":["spark","writes","partitioning","transforms"],"backgroundTag":"unsupported-operation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}