prestodb/presto · error · OrcCorruptionException

Write validation failed: unexpected double range in %s stati

Error message

Write validation failed: unexpected double range in %s statistics

What it means

For double columns, write validation compares DoubleStatistics (min/max) recorded at write time against those read from the file. Objects.equals on DoubleStatistics is exact; any difference (including NaN representation differences across writers) makes validation fail, treating the file as possibly corrupt.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:556

        }
        if (!Objects.equals(actualColumnStatistics.getIntegerStatistics(), expectedColumnStatistics.getIntegerStatistics())) {
            IntegerStatistics actualIntegerStatistics = actualColumnStatistics.getIntegerStatistics();
            IntegerStatistics expectedIntegerStatistics = expectedColumnStatistics.getIntegerStatistics();
            // The sum of the integer stats depends on the order of how we merge them.
            // It is possible the sum can overflow with one order but not in another.
            // Ignore the validation of sum if one of the two sums is null.
            if (actualIntegerStatistics == null ||
                    expectedIntegerStatistics == null ||
                    !Objects.equals(actualIntegerStatistics.getMin(), expectedIntegerStatistics.getMin()) ||
                    !Objects.equals(actualIntegerStatistics.getMax(), expectedIntegerStatistics.getMax()) ||
                    (actualIntegerStatistics.getSum() != null &&
                            expectedIntegerStatistics.getSum() != null &&
                            !Objects.equals(actualIntegerStatistics.getSum(), expectedIntegerStatistics.getSum()))) {
                throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: unexpected integer range in %s statistics", name);
            }
        }
        if (!Objects.equals(actualColumnStatistics.getDoubleStatistics(), expectedColumnStatistics.getDoubleStatistics())) {
            throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: unexpected double range in %s statistics", name);
        }
        StringStatistics expectedStringStatistics = expectedColumnStatistics.getStringStatistics();
        if (expectedStringStatistics != null) {
            expectedStringStatistics = new StringStatistics(
                    minStringTruncateToValidRange(expectedStringStatistics.getMin(), HiveWriterVersion.ORC_HIVE_8732),
                    maxStringTruncateToValidRange(expectedStringStatistics.getMax(), HiveWriterVersion.ORC_HIVE_8732),
                    expectedStringStatistics.isLowerBoundSet(),
                    expectedStringStatistics.isUpperBoundSet(),
                    expectedStringStatistics.getSum());
        }
        StringStatistics actualStringStatistics = actualColumnStatistics.getStringStatistics();
        if (!Objects.equals(actualColumnStatistics.getStringStatistics(), expectedStringStatistics) && expectedStringStatistics != null) {
            // expectedStringStatistics (or the min/max of it) could be null while the actual one might not because
            // expectedStringStatistics is calculated by merging all row group stats in the stripe but the actual one is by scanning each row in the stripe on disk.
            // Merging row group stats can produce nulls given we have string stats limit.
            if (actualStringStatistics == null ||
                    actualStringStatistics.getSum() != expectedStringStatistics.getSum() ||
                    (expectedStringStatistics.getMax() != null && !Objects.equals(actualStringStatistics.getMax(), expectedStringStatistics.getMax())) ||

View on GitHub (pinned to 55bb57d202)

Solutions

Re-write the file with the Presto ORC writer to normalize double statistics
Check whether the file came from another engine (Hive/Spark) and disable orc.write-validation for it
Restore the file if genuine corruption is suspected (verify with storage checksums)
Upgrade Presto if a double statistics equality bug matches your version

Defensive patterns

Strategy: try-catch

Try / catch

try {
    orcBatchReader.nextPage();
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("unexpected double range")) {
        // check for cross-engine NaN/-0.0 serialization differences before declaring corruption
    }
}

Prevention

Avoid mixing ORC writers with different NaN/-0.0 statistics conventions
Re-write cross-engine files with Presto's writer
Keep Presto versions aligned
Enable storage checksums

When it happens

Trigger: validateColumnStatisticsEquivalent finds actualColumnStatistics.getDoubleStatistics() not equal to expectedColumnStatistics.getDoubleStatistics() — min or max differ, or null vs non-null, at any validation level.

Common situations: Files written by other ORC engines that serialize NaN or -0.0 differently; corrupted double columns; floating-point stats computed with different precision by other writers.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/a4f6c046c17de36f. Report an issue: GitHub.