prestodb/presto · error · OrcCorruptionException

Write validation failed: unexpected number of columns in %s

Error message

Write validation failed: unexpected number of columns in %s statistics

What it means

Write validation compares lists of ColumnStatistics (file, stripe, or row group level) positionally between the statistics recorded at write time and those read from the file. If the two lists differ in length, the column count changed between write and read, so validation cannot proceed and OrcCorruptionException is thrown.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:511

            Type type = readColumns.get(column);
            checkArgument(type != null, "statistics validation requires all columns to be read");
            types.add(type);
        }
        return new StatisticsValidation(types.build());
    }

    private static void validateColumnStatisticsEquivalent(
            OrcDataSourceId orcDataSourceId,
            String name,
            List<ColumnStatistics> actualColumnStatistics,
            List<ColumnStatistics> expectedColumnStatistics)
            throws OrcCorruptionException
    {
        requireNonNull(name, "name is null");
        requireNonNull(actualColumnStatistics, "actualColumnStatistics is null");
        requireNonNull(expectedColumnStatistics, "expectedColumnStatistics is null");
        if (actualColumnStatistics.size() != expectedColumnStatistics.size()) {
            throw new OrcCorruptionException(orcDataSourceId, "Write validation failed: unexpected number of columns in %s statistics", name);
        }
        for (int i = 0; i < actualColumnStatistics.size(); i++) {
            ColumnStatistics actual = actualColumnStatistics.get(i);
            ColumnStatistics expected = expectedColumnStatistics.get(i);
            validateColumnStatisticsEquivalent(orcDataSourceId, name + " column " + i, actual, expected);
        }
    }

    private static void validateColumnStatisticsEquivalent(
            OrcDataSourceId orcDataSourceId,
            String name,
            ColumnStatistics actualColumnStatistics,
            ColumnStatistics expectedColumnStatistics)
            throws OrcCorruptionException
    {
        requireNonNull(name, "name is null");
        requireNonNull(actualColumnStatistics, "actualColumnStatistics is null");
        requireNonNull(expectedColumnStatistics, "expectedColumnStatistics is null");

View on GitHub (pinned to 55bb57d202)

Solutions

Verify the file was not altered or rewritten with a different schema after writing
Re-write the file with the Presto ORC writer using the current schema
Upgrade Presto if a known bug in flattened column statistics counting applies to your version
Disable orc.write-validation when reading files produced by other engines or after schema evolution

Defensive patterns

Strategy: validation

Validate before calling

// Skip validation when file schema differs from the written schema (schema evolution)
if (!fileSchema.equals(writerSchema)) {
    session.setProperty("orc_write_validation", false);
}

Try / catch

try {
    orcReader = new OrcReader(...);
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("unexpected number of columns")) {
        // re-read with validation disabled; investigate schema evolution
    }
}

Prevention

Don't apply schema evolution in place to validated ORC files
Write with Presto's writer when validation is on
Verify column count is unchanged after any file transformation
Upgrade Presto to get consistent flattened-column statistics counting

When it happens

Trigger: validateColumnStatisticsEquivalent invoked (from validateFileStatistics, validateStripeStatistics, or validateRowGroupStatistics) with actualColumnStatistics.size() != expectedColumnStatistics.size() — the file's statistics metadata contains a different number of columns than the write-time record.

Common situations: Schema evolution (columns added/dropped) applied to existing ORC files; files rewritten by another engine with different column flattening; corrupted statistics metadata.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/bac7d90d2922a4fb. Report an issue: GitHub.