{"record":{"id":"3f332986ab257e03","repo":"prestodb/presto","slug":"write-validation-failed-unexpected-boolean-counts","errorCode":null,"errorMessage":"Write validation failed: unexpected boolean counts in %s statistics","messagePattern":"Write validation failed: unexpected boolean counts in (.+?) statistics","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java","lineNumber":537,"sourceCode":"\n    private static void validateColumnStatisticsEquivalent(\n            OrcDataSourceId orcDataSourceId,\n            String name,\n            ColumnStatistics actualColumnStatistics,\n            ColumnStatistics expectedColumnStatistics)\n            throws OrcCorruptionException\n    {\n        requireNonNull(name, \"name is null\");\n        requireNonNull(actualColumnStatistics, \"actualColumnStatistics is null\");\n        requireNonNull(expectedColumnStatistics, \"expectedColumnStatistics is null\");\n\n        if (actualColumnStatistics.getNumberOfValues() != expectedColumnStatistics.getNumberOfValues()) {\n            String failureMessage = format(\"Actual Values %s does not match expected values %s\", actualColumnStatistics, expectedColumnStatistics);\n            throw new OrcCorruptionException(orcDataSourceId, \"Write validation failed: %s in %s statistics\", failureMessage, name);\n        }\n\n        if (!Objects.equals(actualColumnStatistics.getBooleanStatistics(), expectedColumnStatistics.getBooleanStatistics())) {\n            throw new OrcCorruptionException(orcDataSourceId, \"Write validation failed: unexpected boolean counts in %s statistics\", name);\n        }\n        if (!Objects.equals(actualColumnStatistics.getIntegerStatistics(), expectedColumnStatistics.getIntegerStatistics())) {\n            IntegerStatistics actualIntegerStatistics = actualColumnStatistics.getIntegerStatistics();\n            IntegerStatistics expectedIntegerStatistics = expectedColumnStatistics.getIntegerStatistics();\n            // The sum of the integer stats depends on the order of how we merge them.\n            // It is possible the sum can overflow with one order but not in another.\n            // Ignore the validation of sum if one of the two sums is null.\n            if (actualIntegerStatistics == null ||\n                    expectedIntegerStatistics == null ||\n                    !Objects.equals(actualIntegerStatistics.getMin(), expectedIntegerStatistics.getMin()) ||\n                    !Objects.equals(actualIntegerStatistics.getMax(), expectedIntegerStatistics.getMax()) ||\n                    (actualIntegerStatistics.getSum() != null &&\n                            expectedIntegerStatistics.getSum() != null &&\n                            !Objects.equals(actualIntegerStatistics.getSum(), expectedIntegerStatistics.getSum()))) {\n                throw new OrcCorruptionException(orcDataSourceId, \"Write validation failed: unexpected integer range in %s statistics\", name);\n            }\n        }\n        if (!Objects.equals(actualColumnStatistics.getDoubleStatistics(), expectedColumnStatistics.getDoubleStatistics())) {","sourceCodeStart":519,"sourceCodeEnd":555,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java#L519-L555","documentation":"For boolean columns, write validation compares BooleanStatistics (true/false counts) recorded at write time against those read from the file. Unequal counts indicate the boolean data in the file differs from what the writer recorded, so the file is treated as corrupt.","triggerScenarios":"validateColumnStatisticsEquivalent finds actualColumnStatistics.getBooleanStatistics() not equal (via Objects.equals) to expectedColumnStatistics.getBooleanStatistics() — true/false/non-null counts differ at file, stripe, or row group level.","commonSituations":"Corrupted boolean columns; files rewritten by other engines computing boolean stats differently; writer/reader version bugs in boolean statistics.","solutions":["Restore or re-write the corrupted file","Compare writer and reader Presto versions; upgrade if a boolean statistics bug was fixed","Verify the file was not modified by another process after writing","Disable orc.write-validation to triage, then regenerate the file"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    orcBatchReader.nextPage();\n} catch (OrcCorruptionException e) {\n    if (e.getMessage().contains(\"unexpected boolean counts\")) {\n        // restore from replica or re-read with orc.write-validation=false\n    }\n}","preventionTips":["Write boolean columns with the Presto ORC writer","Avoid cross-engine rewrites of validated files","Keep Presto versions uniform across clusters","Enable storage-level corruption detection"],"tags":["orc","write-validation","statistics","boolean"],"backgroundTag":"orc-write-validation-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}