prestodb/presto · error · OrcCorruptionException

Write validation failed: unexpected boolean counts in %s sta

Error message

Write validation failed: unexpected boolean counts in %s statistics

What it means

For boolean columns, write validation compares BooleanStatistics (true/false counts) recorded at write time against those read from the file. Unequal counts indicate the boolean data in the file differs from what the writer recorded, so the file is treated as corrupt.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:537

    private static void validateColumnStatisticsEquivalent(
            OrcDataSourceId orcDataSourceId,
            String name,
            ColumnStatistics actualColumnStatistics,
            ColumnStatistics expectedColumnStatistics)
            throws OrcCorruptionException
    {
        requireNonNull(name, "name is null");
        requireNonNull(actualColumnStatistics, "actualColumnStatistics is null");
        requireNonNull(expectedColumnStatistics, "expectedColumnStatistics is null");

        if (actualColumnStatistics.getNumberOfValues() != expectedColumnStatistics.getNumberOfValues()) {
            String failureMessage = format("Actual Values %s does not match expected values %s", actualColumnStatistics, expectedColumnStatistics);
            throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: %s in %s statistics", failureMessage, name);
        }

        if (!Objects.equals(actualColumnStatistics.getBooleanStatistics(), expectedColumnStatistics.getBooleanStatistics())) {
            throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: unexpected boolean counts in %s statistics", name);
        }
        if (!Objects.equals(actualColumnStatistics.getIntegerStatistics(), expectedColumnStatistics.getIntegerStatistics())) {
            IntegerStatistics actualIntegerStatistics = actualColumnStatistics.getIntegerStatistics();
            IntegerStatistics expectedIntegerStatistics = expectedColumnStatistics.getIntegerStatistics();
            // The sum of the integer stats depends on the order of how we merge them.
            // It is possible the sum can overflow with one order but not in another.
            // Ignore the validation of sum if one of the two sums is null.
            if (actualIntegerStatistics == null ||
                    expectedIntegerStatistics == null ||
                    !Objects.equals(actualIntegerStatistics.getMin(), expectedIntegerStatistics.getMin()) ||
                    !Objects.equals(actualIntegerStatistics.getMax(), expectedIntegerStatistics.getMax()) ||
                    (actualIntegerStatistics.getSum() != null &&
                            expectedIntegerStatistics.getSum() != null &&
                            !Objects.equals(actualIntegerStatistics.getSum(), expectedIntegerStatistics.getSum()))) {
                throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: unexpected integer range in %s statistics", name);
            }
        }
        if (!Objects.equals(actualColumnStatistics.getDoubleStatistics(), expectedColumnStatistics.getDoubleStatistics())) {

View on GitHub (pinned to 55bb57d202)

Solutions

Restore or re-write the corrupted file
Compare writer and reader Presto versions; upgrade if a boolean statistics bug was fixed
Verify the file was not modified by another process after writing
Disable orc.write-validation to triage, then regenerate the file

Defensive patterns

Strategy: try-catch

Try / catch

try {
    orcBatchReader.nextPage();
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("unexpected boolean counts")) {
        // restore from replica or re-read with orc.write-validation=false
    }
}

Prevention

Write boolean columns with the Presto ORC writer
Avoid cross-engine rewrites of validated files
Keep Presto versions uniform across clusters
Enable storage-level corruption detection

When it happens

Trigger: validateColumnStatisticsEquivalent finds actualColumnStatistics.getBooleanStatistics() not equal (via Objects.equals) to expectedColumnStatistics.getBooleanStatistics() — true/false/non-null counts differ at file, stripe, or row group level.

Common situations: Corrupted boolean columns; files rewritten by other engines computing boolean stats differently; writer/reader version bugs in boolean statistics.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/3f332986ab257e03. Report an issue: GitHub.