{"record":{"id":"e9a79f5978673be0","repo":"apache/beam","slug":"only-one-of-the-min-max-was-was-null-for-field-columnname","errorCode":null,"errorMessage":"Only one of the min/max was was null, for field {columnName}","messagePattern":"Only one of the min/max was was null, for field (.+?)","errorType":"exception","errorClass":"UnknownPartitionException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/AddFiles.java","lineNumber":603,"sourceCode":"      }\n\n      PartitionKey pk = new PartitionKey(table.spec(), table.schema());\n\n      // read metadata from footer and set partition based on min/max transformed values\n      for (int i = 0; i < fields.size(); i++) {\n        PartitionField field = fields.get(i);\n        Type type = table.schema().findType(field.sourceId());\n        Transform<?, ?> transform = field.transform();\n\n        // Make a best effort estimate by comparing the lower and upper transformed values.\n        // If the transformed values are equal, assume that the DataFile's data safely\n        // aligns with the same partition.\n        ByteBuffer lowerBytes = partitionMetrics.lowerBounds().get(field.sourceId());\n        ByteBuffer upperBytes = partitionMetrics.upperBounds().get(field.sourceId());\n        if (lowerBytes == null && upperBytes == null) {\n          continue;\n        } else if (lowerBytes == null || upperBytes == null) {\n          throw new UnknownPartitionException(\n              \"Only one of the min/max was was null, for field \"\n                  + table.schema().findColumnName(field.sourceId()));\n        }\n        Object lowerTransformedValue = transformValue(transform, type, lowerBytes);\n        Object upperTransformedValue = transformValue(transform, type, upperBytes);\n\n        if (!Objects.deepEquals(lowerTransformedValue, upperTransformedValue)) {\n          // The DataFile contains values that align to different partitions, so we cannot\n          // safely determine a partition.\n          throw new UnknownPartitionException(\n              \"Min and max transformed values were not equal, for column: \" + field.name());\n        }\n\n        pk.set(i, lowerTransformedValue);\n      }\n\n      return pk.toPath();\n    }","sourceCodeStart":585,"sourceCodeEnd":621,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/AddFiles.java#L585-L621","documentation":"AddFiles.getPartitionFromMetrics derives a DataFile's partition from its lower/upper bound metrics. If exactly one of lowerBounds/upperBounds is null for a partition field, the partition cannot be determined safely, so it throws UnknownPartitionException. The file's statistics are incomplete for that column.","triggerScenarios":"Processing a DataFile whose metrics (written by the writing engine) lack either a min or a max for a partition-key column; metrics set to null because the column had all-nulls or stats were dropped (write.metadata.metrics.default=none).","commonSituations":"Tables written by engines with partial statistics collection; partition columns that are entirely NULL in some files; metrics disabled for performance on wide tables; files produced by older Iceberg writers.","solutions":["Rewrite the affected DataFiles with a full rewrite (rewrite_data_files / RewriteDataFiles) so complete min/max stats are produced","Set write.metadata.metrics.default appropriately (e.g. 'truncate' or 'full') on the table before writing new files","Exclude/skip the offending DataFile or route it to a fallback path instead of auto-replicating to a partition","Inspect the file's metrics via Iceberg's MetadataTableColumns to confirm which column lacks bounds"],"exampleFix":"// before\nAddFiles.getDefaultTableLoader(catalogConfig)\n// after (Hive/park route): first run in Spark\nspark.sql(\"CALL catalog.system.rewrite_data_files(table => 'db.tbl')\")","handlingStrategy":"try-catch","validationCode":"Metrics m = file.metrics(); if (m.lowerBounds()==null || m.upperBounds()==null || m.lowerBounds().get(fieldId)==null || m.upperBounds().get(fieldId)==null) skip(file);","typeGuard":"null","tryCatchPattern":"try { partition = getPartitionFromMetrics(...); } catch (UnknownPartitionException e) { routeToFallback(file, e); }","preventionTips":["Enable full metrics on write (write.metadata.metrics.default)","Avoid writing files whose partition columns are all-null","Rewrite legacy files with complete stats","Check table property write.metadata.metrics.default=none"],"tags":["java","iceberg","partitioning","metrics"],"backgroundTag":"missing-required-config-field","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}