{"record":{"id":"247a4c1239f6718b","repo":"apache/beam","slug":"min-and-max-transformed-values-were-not-equal-for-column","errorCode":null,"errorMessage":"Min and max transformed values were not equal, for column: {columnName}","messagePattern":"Min and max transformed values were not equal, for column: (.+?)","errorType":"exception","errorClass":"UnknownPartitionException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/AddFiles.java","lineNumber":613,"sourceCode":"        // Make a best effort estimate by comparing the lower and upper transformed values.\n        // If the transformed values are equal, assume that the DataFile's data safely\n        // aligns with the same partition.\n        ByteBuffer lowerBytes = partitionMetrics.lowerBounds().get(field.sourceId());\n        ByteBuffer upperBytes = partitionMetrics.upperBounds().get(field.sourceId());\n        if (lowerBytes == null && upperBytes == null) {\n          continue;\n        } else if (lowerBytes == null || upperBytes == null) {\n          throw new UnknownPartitionException(\n              \"Only one of the min/max was was null, for field \"\n                  + table.schema().findColumnName(field.sourceId()));\n        }\n        Object lowerTransformedValue = transformValue(transform, type, lowerBytes);\n        Object upperTransformedValue = transformValue(transform, type, upperBytes);\n\n        if (!Objects.deepEquals(lowerTransformedValue, upperTransformedValue)) {\n          // The DataFile contains values that align to different partitions, so we cannot\n          // safely determine a partition.\n          throw new UnknownPartitionException(\n              \"Min and max transformed values were not equal, for column: \" + field.name());\n        }\n\n        pk.set(i, lowerTransformedValue);\n      }\n\n      return pk.toPath();\n    }\n  }\n\n  /**\n   * Writes batches of {@link SerializableDataFile}s (grouped by Partition Spec ID) into {@link\n   * ManifestFile}s.\n   *\n   * <p>Returns the byte-encoded {@link ManifestFile}, to be reconstructed and committed by\n   * downstream {@link CommitManifestFilesDoFn}.\n   */\n  static class CreateManifests","sourceCodeStart":595,"sourceCodeEnd":631,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/AddFiles.java#L595-L631","documentation":"getPartitionFromMetrics requires that, after applying the partition transform, the transformed min and max of each partition column are equal; otherwise the DataFile spans multiple partitions and no single partition can be assigned. The library throws UnknownPartitionException naming the column.","triggerScenarios":"Running AddFiles on a DataFile whose rows span multiple values of a partition column (e.g. a day-granularity file containing rows from two dates under a days(ts) partition transform).","commonSituations":"Importing legacy/unpartitioned files whose contents were not pre-sorted by partition key; overly fine-grained partition transforms (hours(ts)) on coarsely written files; month/day transforms applied to long-lived files.","solutions":["Re-partition/split the source files by the partition key before invoking AddFiles (e.g. sort-and-rewrite with Spark)","Use a coarser partition transform that matches the file granularity so min==max per file","Handle UnknownPartitionException by routing such files to a manual/repair workflow instead of failing the whole job","Rewrite data files first with rewrite_data_files to consolidate values per partition"],"exampleFix":"// before: files contain multiple days, transform days(ts)\nAddFiles.applicableFormat(...)\n// after: coarsen transform to months(ts), or rewrite files per-partition first\nspark.sql(\"CALL cat.system.rewrite_data_files(table => 'db.t', strategy => 'sort', sort_order => 'ts')\")","handlingStrategy":"try-catch","validationCode":"if (lower!=null && upper!=null && !Objects.equals(transform.apply(lower), transform.apply(upper))) routeToFallback(file);","typeGuard":"null","tryCatchPattern":"try { partition = getPartitionFromMetrics(...); } catch (UnknownPartitionException e) { splitFile(file); }","preventionTips":["Sort source data by partition key before AddFiles","Match partition transform granularity to file contents","Rewrite files per-partition before import","Provide a fallback path for multi-partition files"],"tags":["java","iceberg","partitioning","transforms"],"backgroundTag":"invalid-argument-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}