prestodb/presto · error · OrcCorruptionException

Write validation failed: unexpected integer range in %s stat

Error message

Write validation failed: unexpected integer range in %s statistics

What it means

For integer columns, write validation compares IntegerStatistics (min/max, and sum when both are non-null) recorded at write time to those read back. Note the sum check is lenient — sums may differ if merging overflowed in one order but not the other — but min/max and any present sum must match. Mismatches indicate the numeric data or its statistics changed after write.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:552

        }

        if (!Objects.equals(actualColumnStatistics.getBooleanStatistics(), expectedColumnStatistics.getBooleanStatistics())) {
            throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: unexpected boolean counts in %s statistics", name);
        }
        if (!Objects.equals(actualColumnStatistics.getIntegerStatistics(), expectedColumnStatistics.getIntegerStatistics())) {
            IntegerStatistics actualIntegerStatistics = actualColumnStatistics.getIntegerStatistics();
            IntegerStatistics expectedIntegerStatistics = expectedColumnStatistics.getIntegerStatistics();
            // The sum of the integer stats depends on the order of how we merge them.
            // It is possible the sum can overflow with one order but not in another.
            // Ignore the validation of sum if one of the two sums is null.
            if (actualIntegerStatistics == null ||
                    expectedIntegerStatistics == null ||
                    !Objects.equals(actualIntegerStatistics.getMin(), expectedIntegerStatistics.getMin()) ||
                    !Objects.equals(actualIntegerStatistics.getMax(), expectedIntegerStatistics.getMax()) ||
                    (actualIntegerStatistics.getSum() != null &&
                            expectedIntegerStatistics.getSum() != null &&
                            !Objects.equals(actualIntegerStatistics.getSum(), expectedIntegerStatistics.getSum()))) {
                throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: unexpected integer range in %s statistics", name);
            }
        }
        if (!Objects.equals(actualColumnStatistics.getDoubleStatistics(), expectedColumnStatistics.getDoubleStatistics())) {
            throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: unexpected double range in %s statistics", name);
        }
        StringStatistics expectedStringStatistics = expectedColumnStatistics.getStringStatistics();
        if (expectedStringStatistics != null) {
            expectedStringStatistics = new StringStatistics(
                    minStringTruncateToValidRange(expectedStringStatistics.getMin(), HiveWriterVersion.ORC_HIVE_8732),
                    maxStringTruncateToValidRange(expectedStringStatistics.getMax(), HiveWriterVersion.ORC_HIVE_8732),
                    expectedStringStatistics.isLowerBoundSet(),
                    expectedStringStatistics.isUpperBoundSet(),
                    expectedStringStatistics.getSum());
        }
        StringStatistics actualStringStatistics = actualColumnStatistics.getStringStatistics();
        if (!Objects.equals(actualColumnStatistics.getStringStatistics(), expectedStringStatistics) && expectedStringStatistics != null) {
            // expectedStringStatistics (or the min/max of it) could be null while the actual one might not because
            // expectedStringStatistics is calculated by merging all row group stats in the stripe but the actual one is by scanning each row in the stripe on disk.

View on GitHub (pinned to 55bb57d202)

Solutions

Restore or re-write the file — min/max drift means data changed
Enable storage checksums to detect corruption earlier
Upgrade Presto if the mismatch matches a known integer statistics bug (e.g. overflow handling)
Disable orc.write-validation only for triage, then re-write the file

Defensive patterns

Strategy: try-catch

Try / catch

try {
    orcBatchReader.nextPage();
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("unexpected integer range")) {
        // distinguish tolerated sum-overflow diffs from real min/max drift; restore file
    }
}

Prevention

Remember sum-only mismatches from merge-order overflow are tolerated; min/max drift means corruption
Restore files immediately when min/max differs
Upgrade Presto if overflow-related validation bugs affect your version
Use checksummed storage

When it happens

Trigger: validateColumnStatisticsEquivalent detects actual/expected IntegerStatistics are null where expected, min or max differ, or both sums are non-null and differ — at file, stripe, or row group level.

Common situations: Corrupted integer columns; files rewritten by other engines; overflow-related sum differences (which are tolerated) vs genuine min/max drift from data mutation.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/b3f9d637df393687. Report an issue: GitHub.