prestodb/presto · error · OrcCorruptionException

Unexpected column in row group %s in stripe at offset %s

Error message

Unexpected column in row group %s in stripe at offset %s

What it means

Write-validation error: the set of columns present in a row group's statistics differs from the expected set (a column appears in the file that the writer did not record, or vice versa). Fires per row group while validating non-hashed stats; indicates stats/column divergence between writer record and file content.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:303

                throw new OrcCorruptionException(orcDataSourceId, "Unexpected row group count stripe in at offset %s", stripeOffset);
            }
        }

        for (int rowGroupIndex = 0; rowGroupIndex < expectedRowGroupStatistics.size(); rowGroupIndex++) {
            RowGroupStatistics expectedRowGroup = expectedRowGroupStatistics.get(rowGroupIndex);
            if (expectedRowGroup.getValidationMode() != HASHED) {
                Map<Integer, ColumnStatistics> expectedStatistics = expectedRowGroup.getColumnStatistics();
                Map<Integer, ColumnStatistics> actualStatistics = aggregateRowGroupStatisticsFromRowIndex(orcDataSourceId, actualRowGroupStatistics, stripeOffset, rowGroupIndex);

                // remove empty row group stats for empty flat maps
                expectedStatistics = adjustRowGroupStatisticsForFlatMaps(orcDataSourceId, stripeOffset, expectedStatistics);
                actualStatistics = adjustRowGroupStatisticsForFlatMaps(orcDataSourceId, stripeOffset, actualStatistics);

                Set<Integer> actualColumns = actualStatistics.keySet();
                Set<Integer> expectedColumns = expectedStatistics.keySet();

                if (!expectedColumns.equals(actualColumns)) {
                    throw new OrcCorruptionException(orcDataSourceId, "Unexpected column in row group %s in stripe at offset %s", rowGroupIndex, stripeOffset);
                }

                for (Entry<StreamId, List<RowGroupIndex>> entry : actualRowGroupStatistics.entrySet()) {
                    int column = entry.getKey().getColumn();
                    ColumnStatistics actual = actualStatistics.get(column);
                    ColumnStatistics expected = expectedStatistics.get(column);
                    validateColumnStatisticsEquivalent(orcDataSourceId, "Row group " + rowGroupIndex + " in stripe at offset " + stripeOffset, actual, expected);
                }
            }

            if (expectedRowGroup.getValidationMode() != DETAILED) {
                Map<Integer, ColumnStatistics> actualStatistics = aggregateRowGroupStatisticsFromRowIndex(orcDataSourceId, actualRowGroupStatistics, stripeOffset, rowGroupIndex);
                actualStatistics = adjustRowGroupStatisticsForFlatMaps(orcDataSourceId, stripeOffset, actualStatistics);
                RowGroupStatistics actualRowGroup = new RowGroupStatistics(BOTH, actualStatistics);
                RowGroupStatistics adjustedExpectedRowGroup = new RowGroupStatistics(BOTH, adjustRowGroupStatisticsForFlatMaps(orcDataSourceId, stripeOffset, expectedRowGroup.getColumnStatistics()));
                if (adjustedExpectedRowGroup.getHash() != actualRowGroup.getHash()) {
                    throw new OrcCorruptionException(orcDataSourceId, "Checksum mismatch for row group %s in stripe at offset %s", rowGroupIndex, stripeOffset);
                }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Report as an ORC writer bug with validation logs
2. Rewrite the dataset and re-run validation
3. Check for schema evolution applied inconsistently during the write

When it happens

Trigger: Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:303 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/4811d7e39c4b7756. Report an issue: GitHub.