prestodb/presto · error · OrcCorruptionException

Unexpected multiple column statistics for node %s in row gro

Error message

Unexpected multiple column statistics for node %s in row group %s in stripe at offset %s

What it means

During write validation, Presto ORC aggregates the per-row-group ColumnStatistics read from the file's ROW_INDEX stream and compares them to statistics recorded at write time. A regular (non-flattened) column must produce exactly one ColumnStatistics per row group; this error means multiple statistics entries were aggregated for a node that is not a flattened value node, so the ROW_INDEX data is inconsistent with the writer's declared schema. The library throws OrcCorruptionException because the file likely was written by a different/buggy writer or is corrupted.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:389

    private Map<Integer, ColumnStatistics> aggregateRowGroupStatisticsFromRowIndex(
            OrcDataSourceId orcDataSourceId,
            Map<StreamId, List<RowGroupIndex>> actualRowGroupStatistics,
            long stripeOffset,
            int rowGroupIndex)
    {
        // flattened nodes might have multiple ROW_INDEX with the same column, but different sequences
        // aggregate such statistics before the validation
        Map<Integer, List<ColumnStatistics>> actualColumnStatisticsByColumn = new HashMap<>();
        for (Entry<StreamId, List<RowGroupIndex>> entry : actualRowGroupStatistics.entrySet()) {
            int column = entry.getKey().getColumn();
            ColumnStatistics actual = entry.getValue().get(rowGroupIndex).getColumnStatistics();
            List<ColumnStatistics> aggregateStats = actualColumnStatisticsByColumn.computeIfAbsent(column, (key) -> new ArrayList<>());
            aggregateStats.add(actual);

            // Regular nodes have only 1 ColumnStatistics in the ROW_INDEX, flattened nodes
            // might have zero or more column statistics.
            if (aggregateStats.size() != 1 && !flattenedValueNodes.contains(column)) {
                throw new OrcCorruptionException(
                        orcDataSourceId,
                        "Unexpected multiple column statistics for node %s in row group %s in stripe at offset %s",
                        column,
                        rowGroupIndex,
                        stripeOffset);
            }
        }

        return actualColumnStatisticsByColumn.entrySet().stream()
                .collect(Collectors.toMap(Entry::getKey, entry -> mergeColumnStatistics(entry.getValue())));
    }

    private Map<Integer, Integer> getFlattenedKeyToMapNodes(Set<Integer> flattenedNodes, List<OrcType> orcTypes)
    {
        ImmutableMap.Builder<Integer, Integer> keyNodeToMapNode = ImmutableMap.builder();
        flattenedNodes.forEach(mapNodeIndex -> keyNodeToMapNode.put(orcTypes.get(mapNodeIndex).getFieldTypeIndex(0), mapNodeIndex));
        return keyNodeToMapNode.build();
    }

View on GitHub (pinned to 55bb57d202)

Solutions

Verify the ORC file writer version and re-write the file with the same or newer writer version used by Presto
Confirm the file is not truncated/corrupted (checksum the object in storage, re-download)
Upgrade presto-orc/Presto to a version whose flattened-node handling matches the file's writer
If the file is known-good, disable write validation (orc.write-validation=false) to skip verification

Example fix

// before
// reading a file written by an older writer fails validation
Session session = ...; // orc_write_validation=true
// after
session.setProperty("orc_write_validation", false); // skip read-time write validation for legacy files

Defensive patterns

Strategy: validation

Validate before calling

// Before reading, check writer version and skip validation for foreign writers
if (!writerVersion.equals(file.getPostscript().getWriterVersion()) || file.getMetadataValue("write.validation") == null) {
    session.setProperty("orc_write_validation", false);
}

Try / catch

try {
    orcReader = new OrcReader(...);
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("Unexpected multiple column statistics")) {
        // fall back to reading with write validation disabled
    }
}

Prevention

Write ORC files with Presto's own writer when read-side validation is enabled
Keep writer and reader Presto versions aligned
Enable storage checksums to catch corruption before validation does
Don't post-process ORC files with tools that alter ROW_INDEX metadata

When it happens

Trigger: Reading an ORC file where validateRowGroupStatistics (via actualStatistics/aggregateRowGroupStatisticsFromRowIndex) aggregates more than one ROW_INDEX ColumnStatistics for a column whose node is not in flattenedValueNodes — e.g. files written by older ORC writers, nested/complex-type handling differences, or corrupted stripe metadata.

Common situations: Files produced by other ORC writers (Hive, Spark) with different flattened-node metadata; ORC files corrupted in transit; mismatched presto-orc version reading files written by a newer/older writer.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/13c319222defd1f7. Report an issue: GitHub.