{"record":{"id":"b3f9d637df393687","repo":"prestodb/presto","slug":"write-validation-failed-unexpected-integer-range","errorCode":null,"errorMessage":"Write validation failed: unexpected integer range in %s statistics","messagePattern":"Write validation failed: unexpected integer range in (.+?) statistics","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java","lineNumber":552,"sourceCode":"        }\n\n        if (!Objects.equals(actualColumnStatistics.getBooleanStatistics(), expectedColumnStatistics.getBooleanStatistics())) {\n            throw new OrcCorruptionException(orcDataSourceId, \"Write validation failed: unexpected boolean counts in %s statistics\", name);\n        }\n        if (!Objects.equals(actualColumnStatistics.getIntegerStatistics(), expectedColumnStatistics.getIntegerStatistics())) {\n            IntegerStatistics actualIntegerStatistics = actualColumnStatistics.getIntegerStatistics();\n            IntegerStatistics expectedIntegerStatistics = expectedColumnStatistics.getIntegerStatistics();\n            // The sum of the integer stats depends on the order of how we merge them.\n            // It is possible the sum can overflow with one order but not in another.\n            // Ignore the validation of sum if one of the two sums is null.\n            if (actualIntegerStatistics == null ||\n                    expectedIntegerStatistics == null ||\n                    !Objects.equals(actualIntegerStatistics.getMin(), expectedIntegerStatistics.getMin()) ||\n                    !Objects.equals(actualIntegerStatistics.getMax(), expectedIntegerStatistics.getMax()) ||\n                    (actualIntegerStatistics.getSum() != null &&\n                            expectedIntegerStatistics.getSum() != null &&\n                            !Objects.equals(actualIntegerStatistics.getSum(), expectedIntegerStatistics.getSum()))) {\n                throw new OrcCorruptionException(orcDataSourceId, \"Write validation failed: unexpected integer range in %s statistics\", name);\n            }\n        }\n        if (!Objects.equals(actualColumnStatistics.getDoubleStatistics(), expectedColumnStatistics.getDoubleStatistics())) {\n            throw new OrcCorruptionException(orcDataSourceId, \"Write validation failed: unexpected double range in %s statistics\", name);\n        }\n        StringStatistics expectedStringStatistics = expectedColumnStatistics.getStringStatistics();\n        if (expectedStringStatistics != null) {\n            expectedStringStatistics = new StringStatistics(\n                    minStringTruncateToValidRange(expectedStringStatistics.getMin(), HiveWriterVersion.ORC_HIVE_8732),\n                    maxStringTruncateToValidRange(expectedStringStatistics.getMax(), HiveWriterVersion.ORC_HIVE_8732),\n                    expectedStringStatistics.isLowerBoundSet(),\n                    expectedStringStatistics.isUpperBoundSet(),\n                    expectedStringStatistics.getSum());\n        }\n        StringStatistics actualStringStatistics = actualColumnStatistics.getStringStatistics();\n        if (!Objects.equals(actualColumnStatistics.getStringStatistics(), expectedStringStatistics) && expectedStringStatistics != null) {\n            // expectedStringStatistics (or the min/max of it) could be null while the actual one might not because\n            // expectedStringStatistics is calculated by merging all row group stats in the stripe but the actual one is by scanning each row in the stripe on disk.","sourceCodeStart":534,"sourceCodeEnd":570,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java#L534-L570","documentation":"For integer columns, write validation compares IntegerStatistics (min/max, and sum when both are non-null) recorded at write time to those read back. Note the sum check is lenient — sums may differ if merging overflowed in one order but not the other — but min/max and any present sum must match. Mismatches indicate the numeric data or its statistics changed after write.","triggerScenarios":"validateColumnStatisticsEquivalent detects actual/expected IntegerStatistics are null where expected, min or max differ, or both sums are non-null and differ — at file, stripe, or row group level.","commonSituations":"Corrupted integer columns; files rewritten by other engines; overflow-related sum differences (which are tolerated) vs genuine min/max drift from data mutation.","solutions":["Restore or re-write the file — min/max drift means data changed","Enable storage checksums to detect corruption earlier","Upgrade Presto if the mismatch matches a known integer statistics bug (e.g. overflow handling)","Disable orc.write-validation only for triage, then re-write the file"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    orcBatchReader.nextPage();\n} catch (OrcCorruptionException e) {\n    if (e.getMessage().contains(\"unexpected integer range\")) {\n        // distinguish tolerated sum-overflow diffs from real min/max drift; restore file\n    }\n}","preventionTips":["Remember sum-only mismatches from merge-order overflow are tolerated; min/max drift means corruption","Restore files immediately when min/max differs","Upgrade Presto if overflow-related validation bugs affect your version","Use checksummed storage"],"tags":["orc","write-validation","statistics","integer"],"backgroundTag":"orc-write-validation-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}