prestodb/presto · error · OrcCorruptionException

Checksum mismatch for row group %s in stripe at offset %s

Error message

Checksum mismatch for row group %s in stripe at offset %s

What it means

Write-validation error: the checksum recorded for a row group's statistics does not match the checksum computed when re-reading the file. The row group's statistics content changed between write and validation — a corruption or writer-consistency bug.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:320

                if (!expectedColumns.equals(actualColumns)) {
                    throw new OrcCorruptionException(orcDataSourceId, "Unexpected column in row group %s in stripe at offset %s", rowGroupIndex, stripeOffset);
                }

                for (Entry<StreamId, List<RowGroupIndex>> entry : actualRowGroupStatistics.entrySet()) {
                    int column = entry.getKey().getColumn();
                    ColumnStatistics actual = actualStatistics.get(column);
                    ColumnStatistics expected = expectedStatistics.get(column);
                    validateColumnStatisticsEquivalent(orcDataSourceId, "Row group " + rowGroupIndex + " in stripe at offset " + stripeOffset, actual, expected);
                }
            }

            if (expectedRowGroup.getValidationMode() != DETAILED) {
                Map<Integer, ColumnStatistics> actualStatistics = aggregateRowGroupStatisticsFromRowIndex(orcDataSourceId, actualRowGroupStatistics, stripeOffset, rowGroupIndex);
                actualStatistics = adjustRowGroupStatisticsForFlatMaps(orcDataSourceId, stripeOffset, actualStatistics);
                RowGroupStatistics actualRowGroup = new RowGroupStatistics(BOTH, actualStatistics);
                RowGroupStatistics adjustedExpectedRowGroup = new RowGroupStatistics(BOTH, adjustRowGroupStatisticsForFlatMaps(orcDataSourceId, stripeOffset, expectedRowGroup.getColumnStatistics()));
                if (adjustedExpectedRowGroup.getHash() != actualRowGroup.getHash()) {
                    throw new OrcCorruptionException(orcDataSourceId, "Checksum mismatch for row group %s in stripe at offset %s", rowGroupIndex, stripeOffset);
                }
            }
        }
    }

    private Map<Integer, ColumnStatistics> adjustRowGroupStatisticsForFlatMaps(OrcDataSourceId orcDataSourceId, long stripeOffset, Map<Integer, ColumnStatistics> stats)
    {
        if (this.flattenedKeyToMapNodes.isEmpty()) {
            return stats;
        }

        // The only reliable way to detect that a flat map doesn't have non-null & non-empty
        // values is to check the stripe stats for the key node. If it has 0 number of values
        // it means that this flat map writer didn't write anything.
        StripeStatistics stripeStats = this.stripeStatistics.get(stripeOffset);
        if (stripeStats == null) {
            throw new OrcCorruptionException(orcDataSourceId, "Could not find stripe statistics for a stripe at offset %s", stripeOffset);
        }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Re-run the write job; if reproducible, report as an ORC writer bug
2. Verify storage media integrity
3. Ensure no concurrent writers modified the file

When it happens

Trigger: Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcWriteValidation.java:320 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/118cb5cf6828d2cc. Report an issue: GitHub.